morpheus.utils.shared_process_pool.SharedProcessPool
Bases:
<a href="https://docs.python.org/3/library/functions.html#object">object</a>- Attributes
- status
- total_max_workers
Methods
<a href="#morpheus.utils.shared_process_pool.SharedProcessPool.join">join</a>([timeout])Wait until the SharedProcessPool is terminated. <a href="#morpheus.utils.shared_process_pool.SharedProcessPool.reset">reset</a>()Clear all the previous settings and restart the SharedProcessPool. <a href="#morpheus.utils.shared_process_pool.SharedProcessPool.set_usage">set_usage</a>(stage_name, percentage)Set the usage of the SharedProcessPool for a specific stage. <a href="#morpheus.utils.shared_process_pool.SharedProcessPool.start">start</a>()Start the SharedProcessPool that is currently stopped and keep the settings before last shutdown. <a href="#morpheus.utils.shared_process_pool.SharedProcessPool.stop">stop</a>()Stop receiving any new tasks. <a href="#morpheus.utils.shared_process_pool.SharedProcessPool.submit_task">submit_task</a>(stage_name, process_fn, *args, ...)Submit a task to the SharedProcessPool. <a href="#morpheus.utils.shared_process_pool.SharedProcessPool.terminate">terminate</a>()Terminate all processes and shutdown the SharedProcessPool immediately. <a href="#morpheus.utils.shared_process_pool.SharedProcessPool.wait_until_ready">wait_until_ready</a>([timeout])Wait until the SharedProcessPool is running and ready to accept tasks. Wait until the SharedProcessPool is terminated.
- Parameters
- timeout_type_, optional
timeout in seconds to wait for the SharedProcessPool to be terminated, by default None. If None, it will wait indefinitely.
- Raises
- RuntimeError
If is called on a SharedProcessPool that is not stopped.
- TimeoutError
If has been waiting more than the timeout.
Clear all the previous settings and restart the SharedProcessPool.
- Raises
- RuntimeError
If the SharedProcessPool is not already shut down.
Set the usage of the SharedProcessPool for a specific stage.
- Parameters
- stage_namestr
The unique name of the stage.
- percentagefloat
The percentage of the total workers that will be allocated to the stage, should be between 0 and 1.
- Raises
- RuntimeError
If the SharedProcessPool is not running.
- ValueError
If the percentage is not between 0 and 1 or the total usage is greater than 1.
Start the SharedProcessPool that is currently stopped and keep the settings before last shutdown.
- Raises
- RuntimeError
If the SharedProcessPool is not shutdown.
Stop receiving any new tasks.
Submit a task to the SharedProcessPool.
- Parameters
- stage_namestr
The unique name of the stage.
- process_fnCallable
The function to be executed in the process pool.
- argsAny
Arbitrary arguments for the process_fn.
- kwargsAny
Arbitrary keyword arguments for the process_fn.
- Returns
- Task
The task object that includes the result of the process_fn.
- Raises
- RuntimeError
If the SharedProcessPool is not running.
- ValueError
If the stage_name has not been set in the SharedProcessPool.
Terminate all processes and shutdown the SharedProcessPool immediately.
Wait until the SharedProcessPool is running and ready to accept tasks.
- Parameters
- timeout_type_, optional
timeout in seconds to wait for the SharedProcessPool to be ready, by default None. If None, it will wait indefinitely.
- Raises
- RuntimeError
If the SharedProcessPool is not initializing or running.
- TimeoutError
If has been waiting more than the timeout.