Loaders

Module: polygraphy.backend.onnxrt

class SessionFromOnnx(model_bytes, providers=None)[source]

Bases: BaseLoader

Functor that builds an ONNX-Runtime inference session.

Builds an ONNX-Runtime inference session.

Parameters:

  • model_bytes (Union[Union[bytes, str], Callable() -> Union[bytes, str]]) – A serialized ONNX model or a path to a model or a callable that returns one of those.

  • providers (Sequence[str]) – A sequence of execution providers to use in order of priority. Each element of the sequence may be either an exact match or a case-insensitive partial match for the execution providers available in ONNX-Runtime. For example, a value of “cpu” would match the “CPUExecutionProvider”. Defaults to ["cpu"].

call_impl()[source]
Returns:

The inference session.

Return type:

onnxruntime.InferenceSession

__call__(*args, **kwargs)

Invokes the loader by forwarding arguments to call_impl.

Note: call_impl should not be called directly - use this function instead.

session_from_onnx(model_bytes, providers=None)

Immediately evaluated functional variant of SessionFromOnnx .

Builds an ONNX-Runtime inference session.

Parameters:

  • model_bytes (Union[Union[bytes, str], Callable() -> Union[bytes, str]]) – A serialized ONNX model or a path to a model or a callable that returns one of those.

  • providers (Sequence[str]) – A sequence of execution providers to use in order of priority. Each element of the sequence may be either an exact match or a case-insensitive partial match for the execution providers available in ONNX-Runtime. For example, a value of “cpu” would match the “CPUExecutionProvider”. Defaults to ["cpu"].

Returns:

The inference session.

Return type:

onnxruntime.InferenceSession