Find bottleneck of flask application

I just ran into this myself on a Flask app running on a dedicated box from Digital Ocean, so I'll post the solution in case someone else hits this in the future.

I noticed a few days ago that API requests to GitHub were insanely slow, sometimes taking between 10 and 20 seconds. But running my app locally didn't have any issues. I profiled my app, and socket.getaddrinfo was indeed the culprit:

1 15058.431 15058.4310 15058.431 15058.4310 {_socket.getaddrinfo}1 26.545 26.5450 26.545 26.5450 {_ssl.sslwrap}1 23.246 23.2460 23.246 23.2460 {built-in method do_handshake}4 22.387 5.5968 22.387 5.5968 {built-in method read}1 7.632 7.6320 7.632 7.6320 {method 'connect' of '_socket.socket' objects}103 4.995 0.0485 7.131 0.0692 <s/werkzeug/urls.py:374(url_quote)>2 2.459 1.2295 2.578 1.2890 <ssl.py:294(close)>36 1.495 0.0415 10.548 0.2930 <s/werkzeug/routing.py:707(build)>859 1.442 0.0017 1.693 0.0020 {isinstance}.... etc.

Working with Digital Ocean support, and suspecting it was somehow a DNS issue, the working solution was to change (in /etc/resolv.conf)

nameserver 4.2.2.2nameserver 8.8.8.8

to

nameserver 8.8.4.4nameserver 8.8.8.8

For whatever reason, 4.2.2.2 (run by Level3) decided it hated me but for the time being me and Google's DNS are cool.

Update: My colleague Karl suggested I go ahead and set up a local DNS caching server with bind to prevent Google's DNS from hating me as well. This link was super helpful.

I reckon this is a caused by either your remote host is not caching it's DNS lookups or getaddrinfo is slow because of an issue with IPv6.

Try this (a few times) to test if your nameservers seem to be caching:

$ time host easylib.gdufslib.org

and to test whether the lookups is faster when forcing IPv4-only:

import socketsocket.getaddrinfo("easylib.gdufslib.org", 80, socket.AF_INET, 0, socket.SOL_TCP)

If it's caused by the first one you can install a local caching nameserver or just fix so that the existing nameserver caches. If the latter is the cause you can either try to fix it in your code and libs, or disable IPv6 on your host if you don't use it.

I had the same issue in the near past. To see what causes the slowness I decided to see which ones of my endpoints are more being requested and which ones are the bottleneck. In this purpose I have created a tool to analyze requests. it may help you too, have a look at it https://github.com/muatik/flask-profiler