How to activate the process queue in "django-background-tasks"

In case anybody else is trying to get this working - I call a shell script from my Dockerfile that then runs two commands. One to start the webserver (gunicorn or run_server) and one to run the manage.py process_tasks process. The important thing is to run the process_tasks process as a background task.

At the bottom of my Dockerfile I have:CMD ["./run_app.sh"]

And my run_app.sh file looks like:

#!/usr/bin/env bash# start background tasks python manage.py process_tasks &gunicorn --workers=3 app.wsgi -b 0.0.0.0:8080

Note the trailing & from the process tasks command. This enables the shell script to continue processing and run the gunicorn command.

Hope this helps someone.

Well you could make a bash script that makes the process_tasks call and call that script from python

to call the script from python you can use

import subprocesssubprocess.call("process_tasks.sh", shell=True)

as its explained hereRunning bash script from within python

your bash script may look something like this

cd your/virtualenv/directorysource virtualenv/bin/activateexec /path/to/manage.py process_tasks

dont forget to make process_tasks.sh executable

You must run process_tasks command to execute tasks that have been scheduled. There is no other way. Process_tasks checks the db regularly for scheduled tasks and executes them in the background.

In a local environment

Open a terminal, cd into your app folder and launch the command python manage.py process_tasks. If you are using a virtualenv make sure you activate it first.

In a production environment

Option 1: Run a cron job

One solution is to launch the process_tasks command using a cron job. Beware that cron launches a new process at the scheduled interval without caring about the previous processes. You must make sure the process ends by the time the new cron call is scheduled. Otherwise you will be wasting resources on parallel instances of process_tasks. Happily this check is fairly easy to do.

To launch process_tasks for a limited time you can use its duration parameter. For example, to launch it with a life-span of 10 hours you call:

python manage.py process_tasks --duration 36000   

Configure the cron job:In cPanel you can use the Cron Jobs app. Configure the cron job to run hourly and add the following lines as your cron command:

cd /home/username/appname && /home/username/virtualenv/bin/python /home/username/appname/manage.py process_tasks --duration 3540 

This will cd into your app folder and launch the process_tasks command, taking into consideration your virtual environment and using a life-span of 59 minutes for the process_tasks command.

Update the paths to you app & virtual environment accordingly! Adjust the cron job interval and duration parameter to fit your needs!

Benefits:

• If the command fails it will get restarted by the cron job later.
• You don't have to worry about resource/memory leaks too much.

Option 2. Launch command at server startup

Basically you will have to create a script file containing the same launch command and configure your server to run it at startup.

You lose the benefit that your process is restarted from time to time so you will probably have to monitor its health state. If you are going down this path, a good approach is to use supervisord to monitor your process.