Flask App: Update progress bar while function runs
As some others suggested in the comments, the simplest solution is to run your exporting function in another thread, and let your client pull progress information with another request.There are multiple approaches to handle this particular task.Depending on your needs, you might opt for a more or less sophisticated one.
Here's a very (very) minimal example on how to do it with threads:
import randomimport threadingimport timefrom flask import Flaskclass ExportingThread(threading.Thread): def __init__(self): self.progress = 0 super().__init__() def run(self): # Your exporting stuff goes here ... for _ in range(10): time.sleep(1) self.progress += 10exporting_threads = {}app = Flask(__name__)app.debug = True@app.route('/')def index(): global exporting_threads thread_id = random.randint(0, 10000) exporting_threads[thread_id] = ExportingThread() exporting_threads[thread_id].start() return 'task id: #%s' % thread_id@app.route('/progress/<int:thread_id>')def progress(thread_id): global exporting_threads return str(exporting_threads[thread_id].progress)if __name__ == '__main__': app.run()In the index route (/) we spawn a thread for each exporting task, and we return an ID to that task so that the client can retrieve it later with the progress route (/progress/[exporting_thread]).The exporting thread updates its progress value every time it thinks it is appropriate.
On the client side, you would get something like this (this example uses jQuery):
function check_progress(task_id, progress_bar) { function worker() { $.get('progress/' + task_id, function(data) { if (progress < 100) { progress_bar.set_progress(progress) setTimeout(worker, 1000) } }) }}As said, this example is very minimalistic and you should probably go for a slightly more sophisticated approach.Usually, we would store the progress of a particular thread in a database or a cache of some sort, so that we don't rely on a shared structure, hence avoiding most of the memory and concurrency issues my example has.
Redis (https://redis.io) is an in-memory database store that is generally well-suited for this kind of tasks.It integrates ver nicely with Python (https://pypi.python.org/pypi/redis).
I run this simple but educational Flask SSE implementation on localhost. To handle 3rd party (user uploaded) library in GAE:
- Create a directory named
libin your root path. - copy
geventlibrary directory tolibdirectory. Add these lines to your
main.py:import syssys.path.insert(0,'lib')Thats all. If you use
libdirectory from a child folder, use relative reference:sys.path.insert(0, ../../blablabla/lib')
From http://flask.pocoo.org/snippets/116/
# author: oskar.blom@gmail.com## Make sure your gevent version is >= 1.0import geventfrom gevent.wsgi import WSGIServerfrom gevent.queue import Queuefrom flask import Flask, Responseimport time# SSE "protocol" is described here: http://mzl.la/UPFyxYclass ServerSentEvent(object): def __init__(self, data): self.data = data self.event = None self.id = None self.desc_map = { self.data : "data", self.event : "event", self.id : "id" } def encode(self): if not self.data: return "" lines = ["%s: %s" % (v, k) for k, v in self.desc_map.iteritems() if k] return "%s\n\n" % "\n".join(lines)app = Flask(__name__)subscriptions = []# Client code consumes like this.@app.route("/")def index(): debug_template = """ <html> <head> </head> <body> <h1>Server sent events</h1> <div id="event"></div> <script type="text/javascript"> var eventOutputContainer = document.getElementById("event"); var evtSrc = new EventSource("/subscribe"); evtSrc.onmessage = function(e) { console.log(e.data); eventOutputContainer.innerHTML = e.data; }; </script> </body> </html> """ return(debug_template)@app.route("/debug")def debug(): return "Currently %d subscriptions" % len(subscriptions)@app.route("/publish")def publish(): #Dummy data - pick up from request for real data def notify(): msg = str(time.time()) for sub in subscriptions[:]: sub.put(msg) gevent.spawn(notify) return "OK"@app.route("/subscribe")def subscribe(): def gen(): q = Queue() subscriptions.append(q) try: while True: result = q.get() ev = ServerSentEvent(str(result)) yield ev.encode() except GeneratorExit: # Or maybe use flask signals subscriptions.remove(q) return Response(gen(), mimetype="text/event-stream")if __name__ == "__main__": app.debug = True server = WSGIServer(("", 5000), app) server.serve_forever() # Then visit http://localhost:5000 to subscribe # and send messages by visiting http://localhost:5000/publish