nv_ingest_api.util.nim package#

Module contents#

nv_ingest_api.util.nim.create_inference_client(
endpoints: Tuple[str, str],
model_interface: ModelInterface,
auth_token: str | None = None,
infer_protocol: str | None = None,
timeout: float = 120.0,
max_retries: int = 5,
) → NimClient[source]#

Create a NimClient for interfacing with a model inference server.

Parameters:

  • endpoints (tuple) – A tuple containing the gRPC and HTTP endpoints.

  • model_interface (ModelInterface) – The model interface implementation to use.

  • auth_token (str, optional) – Authorization token for HTTP requests (default: None).

  • infer_protocol (str, optional) – The protocol to use (“grpc” or “http”). If not specified, it is inferred from the endpoints.

Returns:

The initialized NimClient.

Return type:

NimClient

Raises:

ValueError – If an invalid infer_protocol is specified.