> This page is for version v1.4.0. > For other versions, use one of these documentation indexes: > - Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt > - dev: https://docs.nvidia.com/dynamo/dev/llms.txt > - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt > - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt > - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt > - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt > - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt > - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt > - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt > - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt > - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt > - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt > - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt > - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt > - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt > - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt > - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt > - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt > - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt > - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt > - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt > For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Python Route Extensions Python route extensions let an external package register additional HTTP routes on the Dynamo Frontend. The routes use the same port as the OpenAI-compatible API and do not require a custom binary or source build. Extensions are opt-in. The Frontend loads only the providers selected with `--frontend-route-extension` or `DYN_FRONTEND_ROUTE_EXTENSIONS`. A selection can be either: - A name registered under the `dynamo.frontend.routes` entry-point group, recommended for packaged extensions. - A direct `module:function` path for development or one-off deployments. Extensions add routes but cannot override built-in routes. A duplicate method and path combination causes Frontend startup to fail. Python route extensions currently support synchronous handlers on static-path `GET` routes only. Path parameters, wildcards, other HTTP methods, and asynchronous handlers are rejected. ## Add a route extension ### Define a route provider A provider is a callable that returns a `FrontendRoute` or an iterable of routes. Each synchronous handler receives a `FrontendExtensionContext` and returns either: - A JSON-serializable body, which produces HTTP 200. - A `FrontendResponse` with an explicit status code and body. ```python # hello_routes.py from dynamo.llm import FrontendExtensionContext, FrontendRoute def _hello(ctx: FrontendExtensionContext): return {"message": "hello world!"} def hello_world_routes(): return [FrontendRoute("GET", "/hello_world", _hello)] ``` ### Register the provider For an installable package, register the provider under the `dynamo.frontend.routes` entry-point group. The entry-point name is the value passed to the Frontend. ```toml # pyproject.toml [project.entry-points."dynamo.frontend.routes"] hello-world = "hello_routes:hello_world_routes" ``` Install the package so the entry point is discoverable: ```bash pip install -e . ``` ### Start the Frontend Select the registered provider by name: ```bash python -m dynamo.frontend --frontend-route-extension hello-world ``` The equivalent environment-variable form is: ```bash DYN_FRONTEND_ROUTE_EXTENSIONS="hello-world" python -m dynamo.frontend ``` Call the added route: ```bash curl localhost:8000/hello_world ``` The response is: ```json {"message": "hello world!"} ``` ## Load an importable provider directly For development or a one-off deployment, pass an importable `module:function` path. The module must be importable, for example by being available on `PYTHONPATH`. ```bash python -m dynamo.frontend \ --frontend-route-extension hello_routes:hello_world_routes ``` A registered entry-point name takes precedence. The `module:function` fallback applies only when no registered name matches and the value contains `:`. ## Return a custom status Return `FrontendResponse` when a handler needs a status other than HTTP 200: ```python from dynamo.llm import FrontendResponse def _health_ready(ctx): body = {"status": "ready" if ctx.has_any_ready_model() else "not ready"} return body if ctx.has_any_ready_model() else FrontendResponse(503, body) ``` `FrontendExtensionContext` provides a read-only view of live Frontend state through these methods: - `is_ready()` - `is_cancelled()` - `has_any_ready_model()` - `is_model_ready_to_serve(name)` - `model_display_names()` - `serving_ready_display_names()` ## Handler limits Handlers execute in a small, dedicated thread pool, separate from inference tokenization. Keep them fast and non-blocking: - A handler that exceeds 30 seconds returns HTTP 503. - A saturated extension pool rejects new requests with the configured overload status code, HTTP 529 by default. - Handler failures return HTTP 500. - Network, disk, or compute-heavy operations can block other extension routes because Python handlers contend for the Global Interpreter Lock (GIL). ## Load multiple providers Repeat `--frontend-route-extension` to load multiple providers: ```bash python -m dynamo.frontend \ --frontend-route-extension health-routes \ --frontend-route-extension metadata-routes ``` For the environment variable, use whitespace-separated values: ```bash DYN_FRONTEND_ROUTE_EXTENSIONS="health-routes metadata-routes" \ python -m dynamo.frontend ``` Provider names are de-duplicated. An unknown name fails startup and reports the available registered extensions. For the complete flag and handler contract reference, see [Frontend Configuration Reference](/dynamo/v1.4.0/reference/components/frontend-configuration#python-route-extensions).