Python + uwsgi - multiprocessing and shared app state
I think there are two routes you could go down.
Have an endpoint "/set_es_cluster" that gets hit by your SNS POST request. This endpoint then sets the key "active_es_cluster", which is read on every ES request by your other processes. The downside of this is that on each ES request you need to do a redis lookup first.
Have a seperate process that gets the POST request specifically (I assume the clusters are not changing often). The purpose of this process is to receive the post request and just have uWSGI gracefully restart your other flask processes.
The advantages of the second option:
- Don't have to hit redis on every request
- Let uWSGI handle the restarts for you (which it does well)
- You already setup the config pulling at runtime anyway so it should "just work" with your existing application