aitune.torch.backend.onnx_runtime_backend
aitune.torch.backend.onnx_runtime_backend
ONNX Runtime backend.
Module Contents
Classes
Functions
Data
API
Bases: enum.Enum
Supported ONNX Runtime execution providers.
Only NVIDIA GPU-backed providers are supported:
CUDA—CUDAExecutionProvider: standard GPU execution.TENSORRT—TensorrtExecutionProviderwithCUDAExecutionProvideras fallback: enables TensorRT engine compilation for maximum GPU throughput.
Bases: Backend[ONNXRuntimeBackendConfig]
Backend that runs Torch or existing ONNX models with ONNX Runtime.
Uses the source path and shapes directly for OnnxModule. Torch modules
are exported to ONNX with dynamic dimensions inferred from graph_spec.
Both paths use the same ONNX Runtime executor.
Workflow:
Load the ONNX Runtime session from disk.
Return every tracked ONNX data file once, relative to the model.
Derive portable values from the checkpointed sample and GraphSpec.
Use an existing ONNX file or export Torch, then load the session.
Create an artifact from the saved interface and recorded shape bounds.
Deactivate backend.
Deploy backend.
Export a Torch module and preserve its output structure.
Return execution providers based on config (no automatic CPU fallback).
CUDA→[("CUDAExecutionProvider", {"device_id": ...})]TENSORRT→["TensorrtExecutionProvider", ("CUDAExecutionProvider", {"device_id": ...})]
Get the output object from the module and sample.
Note: to avoid case where a module returns a reference to the input argument, we make a deep copy of the output object.
Parameters:
PyTorch module
Sample — Sample input to use for model inference.
Returns: Any
The output object from the module.
Run inference through the shared ONNX Runtime I/O binding executor.
Map args/kwargs to session input names using graph_spec locators.
Tensors are returned as-is (preserving their device); I/O binding happens
in the shared run_onnx executor.
Reconstruct original output structure from session output tensors.
Prepare artifacts and output structure from an existing ONNX model.
Run representative samples to initialize the execution provider.
Some execution providers defer work until inference. In particular, the TensorRT execution provider compiles the ONNX graph when it first runs. Exercising a small number of recorded samples makes those failures part of the backend build instead of deferring them until profiling or application inference.
Parameters:
Iterable[ Sample ] — Recorded input samples for the backend build.
Create the deployment record on request, including after deactivation.
Returns the description of the backend.
Creates a backend from a state_dict.
Returns the key of the backend.
Returns the state_dict of the backend.
Bases: BackendConfig
Configuration for ONNXRuntimeBackend.
Parameters:
If True (default), export via torch.onnx.export(dynamo=True)
which calls torch.export.export internally and produces a more accurate
graph (no Python-level tracing limitations). If False, use the classic
trace-based exporter — faster and broader model coverage.
ONNXExecutionProvider | None — ONNX Runtime execution provider to use. When None
the backend defaults to ONNXExecutionProvider.CUDA. Only
ONNXExecutionProvider.CUDA and
ONNXExecutionProvider.TENSORRT are supported.
ONNX opset version passed to torch.onnx.export.
None uses the torch default.
Post init.
Initialise config from a plain dict (e.g. parsed from YAML).
execution_provider may be passed as a string and will be
converted to an ONNXExecutionProvider enum value automatically.
Validate the finalized ONNX interface against the recorded Torch contract.