flask application with background threads
Try this example, tested on Python 3.4.3 / Flask 0.11.1
from flask import Flaskfrom time import sleepfrom concurrent.futures import ThreadPoolExecutor# DOCS https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutorexecutor = ThreadPoolExecutor(2)app = Flask(__name__)@app.route('/jobs')def run_jobs(): executor.submit(some_long_task1) executor.submit(some_long_task2, 'hello', 123) return 'Two jobs were launched in background!'def some_long_task1(): print("Task #1 started!") sleep(10) print("Task #1 is done!")def some_long_task2(arg1, arg2): print("Task #2 started with args: %s %s!" % (arg1, arg2)) sleep(5) print("Task #2 is done!")if __name__ == '__main__': app.run()
Check out Flask-Executor which uses concurrent.futures in the background and makes your life very easy.
from flask_executor import Executorexecutor = Executor(app)@app.route('/someJob')def index(): executor.submit(long_running_job) return 'Scheduled a job'def long_running_job #some long running processing here
This not only runs jobs in the background but gives them access to the app context. It also provides a way to store jobs so users can check back in to get statuses.
The best thing to do for stuff like this is use a message broker. There is some excellent software in the python world meant for doing just this:
- Celery (http://www.celeryproject.org/), and
- RQ (http://python-rq.org/).
Both are excellent choices.
It's almost never a good idea to spawn a thread the way you're doing it, as this can cause issues processing incoming requests, among other things.
If you take a look at the celery or RQ getting started guides, they'll walk you through doing this the proper way!