Capture Heroku SIGTERM in Celery workers to shutdown worker gracefully Capture Heroku SIGTERM in Celery workers to shutdown worker gracefully python python

Capture Heroku SIGTERM in Celery workers to shutdown worker gracefully


python heroku rabbitmq celery sigterm

Starting in version >= 4, Celery comes with a special feature, just for Heroku, that supports this functionality out of the box:

$ REMAP_SIGTERM=SIGQUIT celery -A proj worker -l info

source: https://devcenter.heroku.com/articles/celery-heroku#using-remap_sigterm

python heroku rabbitmq celery sigterm

celery was unfortunately not designed to do clean shutdown. EVER. I mean it. celery workers respond to SIGTERM but if a task is incomplete, the worker processes will wait to finish the task and only then exit. In which case, you can send it SIGKILL if the workers don't shut down in a reasonable time but there will be a loss of information in this case i.e. you may not know which jobs remained incomplete.

python heroku rabbitmq celery sigterm

You can use acks_late or task_acks_late.

Tasks will be acknowledged from queue after task finished execution and not just before. So task will respawn if worker shutdown gracefully.

Recent Posts
How can I color dots in a xy scatterplot according to column value?
How to update a claim in ASP.NET Identity?
What does {0} mean when initializing an object?
Accessing members of items in a JSONArray with Java
How to log SQL statements in Spring Boot?
Powershell Get-WebSite name parameter is ignored
How to detect scroll to bottom of html element
Java synchronized method
How to test controllers with CodeIgniter?
Detect Visual Composer
Matplotlib: Specify format of floats for tick labels
Rails join a list of strings with commas and "and" before the last