Why would running scheduled tasks with Celery be preferable over crontab? Why would running scheduled tasks with Celery be preferable over crontab? python python

Why would running scheduled tasks with Celery be preferable over crontab?


I've been using cron for a production website, and have switched to celery on a current project.I'm far more into celery than cron, here is why:

  • Celery + Celerybeat has finer granularity than cron. Cron cannot run more than once a minute, while celery can (I have a task run every 90 seconds which checks an email queue to send messages, and another which cleans the online users list).
  • A cron line has to call a script or a unique command, with absolute path and user info. Celery calls python functions, no need to write more than code.
  • With celery, to deploy to another machine, you generally just have to pull/copy your code, which is generally in one place. Deploying with cron would need more work (you can automate it but...)
  • I really find celery better suited than cron for routine cleaning (cache, database), and in general, for short tasks. Dumping a database is more a work for cron, however, because you don't want clutter the event queue with too long tasks.
  • Not the least, Celery is easily distributed across machines.


Celery is indicated any time you need to coordinate jobs across multiple machines, ensure jobs run even as machines are added or dropped from a workgroup, have the ability to set expiration times for jobs, define multi-step jobs with graph-style rather than linear dependency flow, or have a single repository of scheduling logic that operates the same across multiple operating systems and versions.