morpheus.utils.shared_process_pool.SharedProcessPool
Bases:
object- Attributes
- status
- total_max_workers
Methods
join([timeout])Wait until the SharedProcessPool is terminated. reset()Clear all the previous settings and restart the SharedProcessPool. set_usage(stage_name, percentage)Set the usage of the SharedProcessPool for a specific stage. start()Start the SharedProcessPool that is currently stopped and keep the settings before last shutdown. stop()Stop receiving any new tasks. submit_task(stage_name, process_fn, *args, ...)Submit a task to the SharedProcessPool. terminate()Terminate all processes and shutdown the SharedProcessPool immediately. wait_until_ready([timeout])Wait until the SharedProcessPool is running and ready to accept tasks. Wait until the SharedProcessPool is terminated.
- Parameters
- timeout_type_, optional
timeout in seconds to wait for the SharedProcessPool to be terminated, by default None. If None, it will wait indefinitely.
- Raises
- RuntimeError
If is called on a SharedProcessPool that is not stopped.
- TimeoutError
If has been waiting more than the timeout.
Clear all the previous settings and restart the SharedProcessPool.
- Raises
- RuntimeError
If the SharedProcessPool is not already shut down.
Set the usage of the SharedProcessPool for a specific stage.
- Parameters
- stage_namestr
The unique name of the stage.
- percentagefloat
The percentage of the total workers that will be allocated to the stage, should be between 0 and 1.
- Raises
- RuntimeError
If the SharedProcessPool is not running.
- ValueError
If the percentage is not between 0 and 1 or the total usage is greater than 1.
Start the SharedProcessPool that is currently stopped and keep the settings before last shutdown.
- Raises
- RuntimeError
If the SharedProcessPool is not shutdown.
Stop receiving any new tasks.
Submit a task to the SharedProcessPool.
- Parameters
- stage_namestr
The unique name of the stage.
- process_fnCallable
The function to be executed in the process pool.
- argsAny
Arbitrary arguments for the process_fn.
- kwargsAny
Arbitrary keyword arguments for the process_fn.
- Returns
- Task
The task object that includes the result of the process_fn.
- Raises
- RuntimeError
If the SharedProcessPool is not running.
- ValueError
If the stage_name has not been set in the SharedProcessPool.
Terminate all processes and shutdown the SharedProcessPool immediately.
Wait until the SharedProcessPool is running and ready to accept tasks.
- Parameters
- timeout_type_, optional
timeout in seconds to wait for the SharedProcessPool to be ready, by default None. If None, it will wait indefinitely.
- Raises
- RuntimeError
If the SharedProcessPool is not initializing or running.
- TimeoutError
If has been waiting more than the timeout.