How do I return flask render_template after the Redis background job is done?
A basic but workable solution (gist):
You could do this by just redirecting from the route which enqueues the job, then have a meta tag refresh that page periodically. First import the required libraries:
from flask import Flask, redirect, url_for, render_template_stringapp = Flask(__name__)from time import sleepfrom rq import Queuefrom rq.job import Jobfrom redis import RedisSet up the rq related connections, and define the function to run:
r = Redis(host='redisserver')q = Queue(connection=r)def slow_func(data): sleep(5) return 'Processed %s' % (data,)Then define a template which can refresh the page every 5 seconds:
template_str='''<html> <head> {% if refresh %} <meta http-equiv="refresh" content="5"> {% endif %} </head> <body>{{result}}</body> </html>'''We'll also make a helper function to return that template with a variable inserted, using flask render_template_string. Notice that refresh defaults to False, if not supplied:
def get_template(data, refresh=False): return render_template_string(template_str, result=data, refresh=refresh)Now make a route which will enqueue our function, get its rq job-id, then return a redirect to the result view with that id. This just takes input in the URL string, but could get that from anywhere:
@app.route('/process/<string:data>')def process(data): job = q.enqueue(slow_func, data) return redirect(url_for('result', id=job.id))Now let's handle the actual result, with the help of the rq.Job object. The logic here could be tweaked, as this will cause a page refresh on all values except "finished":
@app.route('/result/<string:id>')def result(id): job = Job.fetch(id, connection=r) status = job.get_status() if status in ['queued', 'started', 'deferred', 'failed']: return get_template(status, refresh=True) elif status == 'finished': result = job.result # If this is a string, we can simply return it: return get_template(result)If the status is "finished" then job.result will contain the return value of slow_func, so we render this on the page.
This method has the disadvantage of causing several requests to the server, whilst waiting for job completion. The meta refresh tag may be a bit unconventional. If you're sending the request for an update from Javascript, then there are solutions which can send the AJAX request at an interval, though this suffers from the same multiple request problem.
The alternative is to use websockets, or SSE to stream the result of the completed job to the frontend as soon as it completes.
UPDATE: 27 Feb 2021
I decided to have a go at the SSE method of updating the frontend with job status. I learned that rq has native support for updating a meta attribute within the job, by importing rq.get_current_job inside the job, which can then be accessed externally after job refresh.
See the demonstration code for:
A basic example with a progress bar (gist):
