dragonfly.implementations.subprocess_mixin module

Some template vars

Members: SlowSubprocessMixin slow_action Functions: Classes:

A mixin class for endpoints to execute long or indefinite tasks without blocking via subprocess.

Note that a subprocess requires an independent block of memory and is expected to execute independent of the main execution loop. That is, whatever you put into the subprocess should not expect to communicate further with the endpoint itself.

Notes for future me/us: - It is probably desirable/reasonable for the subprocess to be able to send dripline messages, should see if we can pass an provider instance (I think this would clone that object into the subprocess). - One use case is not parallel worker loops but slow-but-finit processes which run once and get a result, should also look into supporting collecting the return object(s) from the workers and having a mechanism for retrieving those if available. - The control_process object is created at __init__ and can thus only be run one time. Should add support for restarting it after it has been killed. This should include possibly updating args/kwargs passed in or similar so that we can support a use case of “i want to change parameter X. i set a new value, then kill the subprocess, then start it back again (or restart if you want to have that defined too)”

Summary

class dragonfly.implementations.subprocess_mixin.SlowSubprocessMixin(worker_function, worker_args=(), worker_kwargs={}, control_function=None)

Bases: object

Mix-in class for managing execution of workers in a subprocess. The default control function simply kills the workers, but a smarter version may be implemented/used instead.

control_function: reference to function to be run, it must have a kwarg ‘halt_event’ which is a multiprocessing.Event. When that event, the function is expected to exit in a timely but graceful fashion. Default=None. If none, use the provided slow_mixin.slow_target method, which will just execute the worker function and terminate() it when told to do so. worker_function: reference to function which actually does the desired work.

basic_control_target(halt_event, target, args=(), kwargs={})

A function which executes a worker function in a subprocess until either it returns, or a halt event is received. When the halt is set, the worker is terminated (the event is polled 10 times per second). There is no need to override this function as the __init__ allows another function name to be passed in (though override would work as well).

halt_event (multiprocessing.Event): event to set() in order to force termination of the subprocess target (function): the function to call in the subprocess args (tuple): items to unpack as positional args to target kwargs (dict): items to unpack as kwargs to target

control_process_is_running()

wrapper to call multiprocessing.Process.is_alive()

join_control_process(timeout=None)

wrapper to call multiprocessing.Process.join(timeout=None)

start_control_process()

wrapper to call multiprocessing.Process.start()

stop_control_process(timeout=0)

Signal the subprocess that it should terminate by setting the halt_event and then terminate it.

timeout (int) [seconds]: number of seconds to allow the process to attempt to exit gracefully, after which terminate() is called if needed.

Note that 0 seconds is valid and results in terminate being called immediately after the halt is set. Note that None is also valid and indicates that the process will be given unlimited time to exit cleanly. This method will join() the process and therefore blocks until the subprocess returns… Do this only with utmost caution

dragonfly.implementations.subprocess_mixin.slow_action(this_time)

A dumb function that does some prints and sleeps to simulate something slow but useful.

End of modules condition

Data:

End of data condition