Production ready Python apps on Kubernetes

We use Twisted's WSGI server with 30 threads and it's been solid for our Django application. Keeps to a single process per pod model which more closely matches Kubernetes' expectations, as you mentioned. Yes, the GIL means only one of those 30 threads can be running Python code at time, but as with most webapps, most of those threads are blocked on I/O (usually waiting for a response from the database) the vast majority of the time. Then run multiple replicas on top of that both for redundancy and to give you true concurrency at whatever level you need (we usually use 4-8 depending on the site traffic, some big ones are up to 16).

I have exactly the same problem with a python deployment running the Flask application. Most api calls are handled in a matter of seconds, but there are some cpu intensive requests that acquire GIL for 2 minutes.... The pod keep accepting requests, ignores the configured timeouts, ignores a closed connection by the user; then after 1 minute of liveness probes failing, the pod is restarted by kubelet.

So 1 fat request can dramatically drop the availability.

I see two different solutions:1) have a separate deployment that will host only long running api calls; configure ingress to route requests between these two deployments; 2) using multiprocessing handle liveness/readyness probes in a main process, every other request must be handled in the child process;

There are pros and cons for each solution, maybe I will need a combination of both. Also if I need a steady flow of prometheus metrics, I might need to create a proxy server on the application layer (1 more container on the same pod). Also need to configure ingress to have a single upstream connection to python pods, so that long running request will be queued, whereas short ones will be processed concurrently (yep, python, concurrency, good joke). Not sure tho it will scale well with HPA.

So yeah, running production ready python rest api server on kubernetes is not a peas of cake. Go and java have a much better ecosystem for microservice applications.

PShere is a good article that shows that there is no need to run your app in kubernetes with WSGIhttps://techblog.appnexus.com/beyond-hello-world-modern-asynchronous-python-in-kubernetes-f2c4ecd4a38d

PPSIm considering to use prometheus exporter for flask. Looks better than running a python client in a separate thread;https://github.com/rycus86/prometheus_flask_exporter