Rollout strategy

[Xowap]

Hi,

I'm wondering what is the recommended rollout strategy. For example, I work with DigitalOcean's PaaS. When I do a new deployment, the system won't necessarily wait until my worker gracefully shuts down before sending it a SIGKILL.

As a result I often end up with stalled tasks that were interrupted while running.

A way that I can imaging solving this is by having something like this:

1. A command tells all running workers to stop starting tasks
2. Another command waits until no tasks are doing anymore (the command would run as a pre-deploy command and can delay the deployment process as needed)
3. Once all tasks are done, then the PaaS will SIGTERM the worker(s)
4. And finally a new worker process is deployed

This is how I imagine things. I'm not sure what is the "canonical" way to deal with this problem?

Thanks!