nv_ingest.framework.orchestration.execution package#

Submodules#

nv_ingest.framework.orchestration.execution.helpers module#

Helper functions for pipeline execution configuration.

This module contains generic helper functions for converting individual parameters into structured configuration objects, supporting the declarative execution architecture.

nv_ingest.framework.orchestration.execution.helpers.create_execution_options(
block: bool,
stdout: TextIO | None,
stderr: TextIO | None,
) → ExecutionOptions[source]#

Create execution options object from individual parameters.

This function converts individual execution parameters into a structured ExecutionOptions object for declarative processing.

Parameters:

  • block (bool) – Whether to block until pipeline completion.

  • stdout (Optional[TextIO]) – Output stream for subprocess redirection.

  • stderr (Optional[TextIO]) – Error stream for subprocess redirection.

Returns:

Structured options object containing the provided values.

Return type:

ExecutionOptions

nv_ingest.framework.orchestration.execution.helpers.create_runtime_overrides(
disable_dynamic_scaling: bool | None,
dynamic_memory_threshold: float | None,
) → PipelineRuntimeOverrides[source]#

Create runtime override object from individual parameters.

This function converts the individual override parameters into a structured PipelineRuntimeOverrides object for declarative processing.

Parameters:

  • disable_dynamic_scaling (Optional[bool]) – Dynamic scaling override value.

  • dynamic_memory_threshold (Optional[float]) – Memory threshold override value.

Returns:

Structured override object containing the provided values.

Return type:

PipelineRuntimeOverrides

nv_ingest.framework.orchestration.execution.helpers.select_execution_strategy(
run_in_subprocess: bool,
) → ProcessExecutionStrategy[source]#

Select appropriate execution strategy based on parameters.

This function encapsulates the logic for choosing between in-process and subprocess execution strategies.

Parameters:

run_in_subprocess (bool) – Whether to run in a subprocess.

Returns:

Configured execution strategy instance.

Return type:

ProcessExecutionStrategy

nv_ingest.framework.orchestration.execution.options module#

Data classes for pipeline execution configuration and options.

This module defines declarative data structures for configuring pipeline execution, replacing imperative parameter passing with structured configuration objects.

class nv_ingest.framework.orchestration.execution.options.ExecutionOptions(
block: bool = True,
stdout: TextIO | None = None,
stderr: TextIO | None = None,
)[source]#

Bases: object

Options controlling pipeline execution behavior.

These options determine how the pipeline is executed (blocking vs non-blocking) and where output is directed for subprocess execution.

block#

If True, blocks until pipeline completes. If False, returns immediately with a control interface.

Type:

bool

stdout#

Stream for subprocess stdout redirection. Only used when run_in_subprocess=True. If None, redirected to /dev/null.

Type:

Optional[TextIO]

stderr#

Stream for subprocess stderr redirection. Only used when run_in_subprocess=True. If None, redirected to /dev/null.

Type:

Optional[TextIO]

block: bool = True#
stderr: TextIO | None = None#
stdout: TextIO | None = None#
class nv_ingest.framework.orchestration.execution.options.ExecutionResult(
interface: RayPipelineInterface | RayPipelineSubprocessInterface | None,
elapsed_time: float | None = None,
)[source]#

Bases: object

Result of pipeline execution containing interface and timing information.

This class encapsulates the results of pipeline execution and provides methods to convert to the legacy return format for backward compatibility.

interface#

Pipeline control interface. None for blocking subprocess execution.

Type:

Union[RayPipelineInterface, RayPipelineSubprocessInterface, None]

elapsed_time#

Total execution time in seconds. Only set for blocking execution.

Type:

Optional[float]

elapsed_time: float | None = None#
get_return_value() → RayPipelineInterface | float | RayPipelineSubprocessInterface[source]#

Convert to legacy return format for backward compatibility.

Returns:

  • If blocking execution: returns elapsed time (float)

  • If non-blocking execution: returns pipeline interface

Return type:

Union[RayPipelineInterface, float, RayPipelineSubprocessInterface]

interface: RayPipelineInterface | RayPipelineSubprocessInterface | None#
class nv_ingest.framework.orchestration.execution.options.PipelineRuntimeOverrides(
disable_dynamic_scaling: bool | None = None,
dynamic_memory_threshold: float | None = None,
)[source]#

Bases: object

Runtime parameter overrides for pipeline configuration.

These overrides are applied to the base pipeline configuration to customize runtime behavior without modifying the source config.

disable_dynamic_scaling#

Override for dynamic scaling behavior. If provided, overrides the pipeline config’s disable_dynamic_scaling setting.

Type:

Optional[bool]

dynamic_memory_threshold#

Override for memory threshold used in dynamic scaling decisions. Must be between 0.0 and 1.0 if provided.

Type:

Optional[float]

disable_dynamic_scaling: bool | None = None#
dynamic_memory_threshold: float | None = None#

Module contents#