nv_ingest.framework.orchestration.ray.stages.sinks package#

Submodules#

nv_ingest.framework.orchestration.ray.stages.sinks.default_drain module#

nv_ingest.framework.orchestration.ray.stages.sinks.message_broker_task_sink module#

pydantic model nv_ingest.framework.orchestration.ray.stages.sinks.message_broker_task_sink.BaseBrokerClientConfig[source]#

Bases: BaseModel

Base configuration common to all broker clients.

Show JSON schema
{
   "title": "BaseBrokerClientConfig",
   "description": "Base configuration common to all broker clients.",
   "type": "object",
   "properties": {
      "host": {
         "description": "Hostname or IP address of the message broker.",
         "title": "Host",
         "type": "string"
      },
      "port": {
         "description": "Port number of the message broker.",
         "title": "Port",
         "type": "integer"
      },
      "max_retries": {
         "default": 5,
         "description": "Maximum number of connection retries.",
         "minimum": 0,
         "title": "Max Retries",
         "type": "integer"
      },
      "max_backoff": {
         "default": 5.0,
         "description": "Maximum backoff delay in seconds between retries.",
         "exclusiveMinimum": 0,
         "title": "Max Backoff",
         "type": "number"
      },
      "connection_timeout": {
         "default": 30.0,
         "description": "Connection timeout in seconds.",
         "exclusiveMinimum": 0,
         "title": "Connection Timeout",
         "type": "number"
      }
   },
   "required": [
      "host",
      "port"
   ]
}

Fields:
field connection_timeout: float = 30.0#

Connection timeout in seconds.

Constraints:

  • gt = 0

field host: str [Required]#

Hostname or IP address of the message broker.

field max_backoff: float = 5.0#

Maximum backoff delay in seconds between retries.

Constraints:

  • gt = 0

field max_retries: int = 5#

Maximum number of connection retries.

Constraints:

  • ge = 0

field port: int [Required]#

Port number of the message broker.

pydantic model nv_ingest.framework.orchestration.ray.stages.sinks.message_broker_task_sink.BrokerParamsRedis[source]#

Bases: BaseModel

Specific parameters for Redis broker_params.

Show JSON schema
{
   "title": "BrokerParamsRedis",
   "description": "Specific parameters for Redis broker_params.",
   "type": "object",
   "properties": {
      "db": {
         "default": 0,
         "title": "Db",
         "type": "integer"
      },
      "use_ssl": {
         "default": false,
         "title": "Use Ssl",
         "type": "boolean"
      }
   }
}

Fields:
field db: int = 0#
field use_ssl: bool = False#
pydantic model nv_ingest.framework.orchestration.ray.stages.sinks.message_broker_task_sink.MessageBrokerTaskSinkConfig[source]#

Bases: BaseModel

Configuration for the MessageBrokerTaskSinkStage.

broker_client#

Configuration parameters for connecting to the message broker. The specific schema is determined by the ‘client_type’ field.

Type:

Union[RedisClientConfig, SimpleClientConfig]

poll_interval#

The polling interval (in seconds) for processing messages. Defaults to 0.1.

Type:

float, optional

Show JSON schema
{
   "title": "MessageBrokerTaskSinkConfig",
   "description": "Configuration for the MessageBrokerTaskSinkStage.\n\nAttributes\n----------\nbroker_client : Union[RedisClientConfig, SimpleClientConfig]\n    Configuration parameters for connecting to the message broker.\n    The specific schema is determined by the 'client_type' field.\npoll_interval : float, optional\n    The polling interval (in seconds) for processing messages. Defaults to 0.1.",
   "type": "object",
   "properties": {
      "broker_client": {
         "discriminator": {
            "mapping": {
               "redis": "#/$defs/RedisClientConfig",
               "simple": "#/$defs/SimpleClientConfig"
            },
            "propertyName": "client_type"
         },
         "oneOf": [
            {
               "$ref": "#/$defs/RedisClientConfig"
            },
            {
               "$ref": "#/$defs/SimpleClientConfig"
            }
         ],
         "title": "Broker Client"
      },
      "poll_interval": {
         "default": 0.1,
         "exclusiveMinimum": 0,
         "title": "Poll Interval",
         "type": "number"
      }
   },
   "$defs": {
      "BrokerParamsRedis": {
         "description": "Specific parameters for Redis broker_params.",
         "properties": {
            "db": {
               "default": 0,
               "title": "Db",
               "type": "integer"
            },
            "use_ssl": {
               "default": false,
               "title": "Use Ssl",
               "type": "boolean"
            }
         },
         "title": "BrokerParamsRedis",
         "type": "object"
      },
      "RedisClientConfig": {
         "description": "Configuration specific to the Redis client.",
         "properties": {
            "host": {
               "description": "Hostname or IP address of the message broker.",
               "title": "Host",
               "type": "string"
            },
            "port": {
               "description": "Port number of the message broker.",
               "title": "Port",
               "type": "integer"
            },
            "max_retries": {
               "default": 5,
               "description": "Maximum number of connection retries.",
               "minimum": 0,
               "title": "Max Retries",
               "type": "integer"
            },
            "max_backoff": {
               "default": 5.0,
               "description": "Maximum backoff delay in seconds between retries.",
               "exclusiveMinimum": 0,
               "title": "Max Backoff",
               "type": "number"
            },
            "connection_timeout": {
               "default": 30.0,
               "description": "Connection timeout in seconds.",
               "exclusiveMinimum": 0,
               "title": "Connection Timeout",
               "type": "number"
            },
            "client_type": {
               "const": "redis",
               "description": "Specifies the client type as Redis.",
               "title": "Client Type",
               "type": "string"
            },
            "broker_params": {
               "$ref": "#/$defs/BrokerParamsRedis",
               "description": "Redis-specific parameters like db and ssl."
            }
         },
         "required": [
            "host",
            "port",
            "client_type"
         ],
         "title": "RedisClientConfig",
         "type": "object"
      },
      "SimpleClientConfig": {
         "description": "Configuration specific to the Simple client.",
         "properties": {
            "host": {
               "description": "Hostname or IP address of the message broker.",
               "title": "Host",
               "type": "string"
            },
            "port": {
               "description": "Port number of the message broker.",
               "title": "Port",
               "type": "integer"
            },
            "max_retries": {
               "default": 5,
               "description": "Maximum number of connection retries.",
               "minimum": 0,
               "title": "Max Retries",
               "type": "integer"
            },
            "max_backoff": {
               "default": 5.0,
               "description": "Maximum backoff delay in seconds between retries.",
               "exclusiveMinimum": 0,
               "title": "Max Backoff",
               "type": "number"
            },
            "connection_timeout": {
               "default": 30.0,
               "description": "Connection timeout in seconds.",
               "exclusiveMinimum": 0,
               "title": "Connection Timeout",
               "type": "number"
            },
            "client_type": {
               "const": "simple",
               "description": "Specifies the client type as Simple.",
               "title": "Client Type",
               "type": "string"
            },
            "broker_params": {
               "anyOf": [
                  {
                     "additionalProperties": true,
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": {},
               "description": "Optional parameters for Simple client (currently unused).",
               "title": "Broker Params"
            }
         },
         "required": [
            "host",
            "port",
            "client_type"
         ],
         "title": "SimpleClientConfig",
         "type": "object"
      }
   },
   "required": [
      "broker_client"
   ]
}

Fields:
field broker_client: RedisClientConfig | SimpleClientConfig [Required]#
field poll_interval: float = 0.1#
Constraints:

  • gt = 0

pydantic model nv_ingest.framework.orchestration.ray.stages.sinks.message_broker_task_sink.RedisClientConfig[source]#

Bases: BaseBrokerClientConfig

Configuration specific to the Redis client.

Show JSON schema
{
   "title": "RedisClientConfig",
   "description": "Configuration specific to the Redis client.",
   "type": "object",
   "properties": {
      "host": {
         "description": "Hostname or IP address of the message broker.",
         "title": "Host",
         "type": "string"
      },
      "port": {
         "description": "Port number of the message broker.",
         "title": "Port",
         "type": "integer"
      },
      "max_retries": {
         "default": 5,
         "description": "Maximum number of connection retries.",
         "minimum": 0,
         "title": "Max Retries",
         "type": "integer"
      },
      "max_backoff": {
         "default": 5.0,
         "description": "Maximum backoff delay in seconds between retries.",
         "exclusiveMinimum": 0,
         "title": "Max Backoff",
         "type": "number"
      },
      "connection_timeout": {
         "default": 30.0,
         "description": "Connection timeout in seconds.",
         "exclusiveMinimum": 0,
         "title": "Connection Timeout",
         "type": "number"
      },
      "client_type": {
         "const": "redis",
         "description": "Specifies the client type as Redis.",
         "title": "Client Type",
         "type": "string"
      },
      "broker_params": {
         "$ref": "#/$defs/BrokerParamsRedis",
         "description": "Redis-specific parameters like db and ssl."
      }
   },
   "$defs": {
      "BrokerParamsRedis": {
         "description": "Specific parameters for Redis broker_params.",
         "properties": {
            "db": {
               "default": 0,
               "title": "Db",
               "type": "integer"
            },
            "use_ssl": {
               "default": false,
               "title": "Use Ssl",
               "type": "boolean"
            }
         },
         "title": "BrokerParamsRedis",
         "type": "object"
      }
   },
   "required": [
      "host",
      "port",
      "client_type"
   ]
}

Fields:
field broker_params: BrokerParamsRedis [Optional]#

Redis-specific parameters like db and ssl.

field client_type: Literal['redis'] [Required]#

Specifies the client type as Redis.

pydantic model nv_ingest.framework.orchestration.ray.stages.sinks.message_broker_task_sink.SimpleClientConfig[source]#

Bases: BaseBrokerClientConfig

Configuration specific to the Simple client.

Show JSON schema
{
   "title": "SimpleClientConfig",
   "description": "Configuration specific to the Simple client.",
   "type": "object",
   "properties": {
      "host": {
         "description": "Hostname or IP address of the message broker.",
         "title": "Host",
         "type": "string"
      },
      "port": {
         "description": "Port number of the message broker.",
         "title": "Port",
         "type": "integer"
      },
      "max_retries": {
         "default": 5,
         "description": "Maximum number of connection retries.",
         "minimum": 0,
         "title": "Max Retries",
         "type": "integer"
      },
      "max_backoff": {
         "default": 5.0,
         "description": "Maximum backoff delay in seconds between retries.",
         "exclusiveMinimum": 0,
         "title": "Max Backoff",
         "type": "number"
      },
      "connection_timeout": {
         "default": 30.0,
         "description": "Connection timeout in seconds.",
         "exclusiveMinimum": 0,
         "title": "Connection Timeout",
         "type": "number"
      },
      "client_type": {
         "const": "simple",
         "description": "Specifies the client type as Simple.",
         "title": "Client Type",
         "type": "string"
      },
      "broker_params": {
         "anyOf": [
            {
               "additionalProperties": true,
               "type": "object"
            },
            {
               "type": "null"
            }
         ],
         "default": {},
         "description": "Optional parameters for Simple client (currently unused).",
         "title": "Broker Params"
      }
   },
   "required": [
      "host",
      "port",
      "client_type"
   ]
}

Fields:
field broker_params: Dict[str, Any] | None = {}#

Optional parameters for Simple client (currently unused).

field client_type: Literal['simple'] [Required]#

Specifies the client type as Simple.

Module contents#