How to share in memory resources between Flask methods when deploying with Gunicorn

You can use preloading.

This will allow you to create the data structure ahead of time, then fork each request handling process. This works because of copy-on-write and the knowledge that you are only reading from the large data structure.

Note: Although this will work, it should probably only be used for very small apps or in a development environment. I think the more production-friendly way of doing this would be to queue up these calculations as tasks on the backend since they will be long-running. You can then notify users of the completed state.

Here is a little snippet to see the difference of preloading.

# app.pyimport flaskapp = flask.Flask(__name__)def load_data():    print('calculating some stuff')    return {'big': 'data'}@app.route('/')def index():    return repr(data)data = load_data()

Running with gunicorn app:app --workers 2:

[2017-02-24 09:01:01 -0500] [38392] [INFO] Starting gunicorn 19.6.0[2017-02-24 09:01:01 -0500] [38392] [INFO] Listening at: http://127.0.0.1:8000 (38392)[2017-02-24 09:01:01 -0500] [38392] [INFO] Using worker: sync[2017-02-24 09:01:01 -0500] [38395] [INFO] Booting worker with pid: 38395[2017-02-24 09:01:01 -0500] [38396] [INFO] Booting worker with pid: 38396calculating some stuffcalculating some stuff

And running with gunicorn app:app --workers 2 --preload:

calculating some stuff[2017-02-24 09:01:06 -0500] [38403] [INFO] Starting gunicorn 19.6.0[2017-02-24 09:01:06 -0500] [38403] [INFO] Listening at: http://127.0.0.1:8000 (38403)[2017-02-24 09:01:06 -0500] [38403] [INFO] Using worker: sync[2017-02-24 09:01:06 -0500] [38406] [INFO] Booting worker with pid: 38406[2017-02-24 09:01:06 -0500] [38407] [INFO] Booting worker with pid: 38407