requests: Requests is slow to upload large files

I am using requests to POST data to a server. In testing with a 14.4 MB file, requests took 16.5 seconds and cURL took 3.3 seconds. The following image shows the 3 tests I ran. The first bump is using requests with MultipartEncoder from requests-toolbelt. The second bump is just using requests. The last bump is cURL. request_speed

I am using Python 3.4.0, requests 2.3.0, and requests-toolbelt 0.3.1 on Windows 7 x64.

Are there any settings I can tune to speed up requests?

About this issue

Original URL
State: closed
Created 10 years ago
Comments: 21 (14 by maintainers)

Most upvoted comments

@kpflugshaupt I’ve managed to change it like this:

from http.client import HTTPConnection
HTTPConnection.__init__.__defaults__ = tuple(
    x if x != 8192 else 64*1024
    for x in HTTPConnection.__init__.__defaults__
)

happy-monk on Oct 21, 2020