nv_ingest.util.message_brokers package

Subpackages

• nv_ingest.util.message_brokers.kafka package
  • Module contents
• nv_ingest.util.message_brokers.redis package
  • Submodules
  • nv_ingest.util.message_brokers.redis.redis_client module
    • RedisClient
      • client
      • fetch_message()
      • get_client()
      • max_retries
      • ping()
      • submit_message()
  • Module contents
• nv_ingest.util.message_brokers.simple_message_broker package
  • Submodules
  • nv_ingest.util.message_brokers.simple_message_broker.broker module
    • SimpleMessageBroker
      • allow_reuse_address
    • SimpleMessageBrokerHandler
      • handle()
  • nv_ingest.util.message_brokers.simple_message_broker.ordered_message_queue module
    • OrderedMessageQueue
      • acknowledge()
      • can_push()
      • empty()
      • full()
      • pop()
      • push()
      • qsize()
      • return_message()
  • nv_ingest.util.message_brokers.simple_message_broker.simple_client module
    • SimpleClient
      • fetch_message()
      • get_client()
      • ping()
      • size()
      • submit_message()
  • Module contents
    • ResponseSchema
      • model_config
      • response
      • response_code
      • response_reason
      • trace_id
      • transaction_id
    • SimpleClient
      • fetch_message()
      • get_client()
      • ping()
      • size()
      • submit_message()
    • SimpleMessageBroker
      • allow_reuse_address

Submodules

nv_ingest.util.message_brokers.client_base module

class nv_ingest.util.message_brokers.client_base.MessageBrokerClientBase(

host: str,

port: int,

db: int = 0,

max_retries: int = 0,

max_backoff: int = 32,

connection_timeout: int = 300,

max_pool_size: int = 128,

use_ssl: bool = False,

)

Bases: ABC

Abstract base class for a messaging client to interface with various messaging systems.

Provides a standard interface for sending and receiving messages with connection management and retry logic.

abstract fetch_message(

job_index: str,

timeout: float = 0,

) → str

Fetches a message from the specified queue with retries on failure.

Parameters:

• job_index (str) – The index of the job to fetch the message for.

• timeout (float) – The timeout in seconds for blocking until a message is available.

Returns:

The fetched message, or None if no message could be fetched.

abstract get_client()

Returns the client instance, reconnecting if necessary.

Returns:

The client instance.

abstract ping() → bool

Checks if the server is responsive.

Returns:

True if the server responds to a ping, False otherwise.

abstract submit_message(

channel_name: str,

message: str,

) → str

Submits a message to a specified queue with retries on failure.

Parameters:

• channel_name (str) – The name of the queue to submit the message to.

• message (str) – The message to submit.

Module contents