How to limit download rate of HTTP requests in requests python library?
There are several approaches to rate limiting; one of them is token bucket, for which you can find a recipe here and another one here.
Usually you would want to do throttling or rate limiting on socket.send() and socket.recv(). You could play with socket-throttle and see if it does what you need.
This is not to be confused with x-ratelimit rate limiting response headers, which are related to a number of requests rather than a download / transfer rate.
No built-in support but, it is possible to use stream api.
>>> import requests>>> import time>>> req = requests.request('GET', 'https://httpbin.org/get', stream=True)>>> for data in req.iter_content(chunk_size=1024):... time.sleep(0.001)...In advanced usage there is written that its's allow you to retrieve smaller quantities of the response at a time.
In my network the example above (leading to a several GB large file) without sleep had bandwidth 17.4 MBps and with sleep 1 ms 2.5 MB/s.