requests: Error decoding chunked response in _update_chunk_length: ValueError: invalid literal for int() with base 16: b'HTTP/1.1 200 OK\r\n'
Decoding a response with chunked encoding fails because it seems to be reading from the HTTP status line instead of the start of the body.
Expected Result
Successfully return a 200 response object.
Actual Result
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/response.py", line 535, in _update_chunk_length
self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b'HTTP/1.1 200 OK\r\n'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/response.py", line 298, in _error_catcher
yield
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/response.py", line 590, in read_chunked
self._update_chunk_length()
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/response.py", line 539, in _update_chunk_length
raise httplib.IncompleteRead(line)
http.client.IncompleteRead: IncompleteRead(17 bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/models.py", line 719, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/response.py", line 428, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/response.py", line 618, in read_chunked
self._original_response.close()
File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/response.py", line 316, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(17 bytes read)', IncompleteRead(17 bytes read))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 641, in send
r.content
File "/usr/lib/python3.6/site-packages/requests/models.py", line 797, in content
self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
File "/usr/lib/python3.6/site-packages/requests/models.py", line 722, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(17 bytes read)', IncompleteRead(17 bytes read))
Reproduction Steps
import requests
requests.get('https://my.virginmoney.com.au/AUVMA/JSO/signon/DisplayUsernameSignon.do')
System Information
requests 2.13.0 urllib3 1.20 Python 3.6.2 Fedora 26
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 6
- Comments: 25 (5 by maintainers)
i’m facing same kind of issue. any idea how to bypass this?
I’m running into this exact issue as well, with an identical traceback as https://github.com/requests/requests/issues/4248#issuecomment-398007524. I’ve tried reverting requests as far back as 2.9.1, and as far forward as the current release (2.20.1), with a few different combinations of urllib3 versions in each case - they all give the exact same traceback.
Is there a release which we can use until this is fixed, or do we have to patch our requests with the above corrected def?
I solved this problem by adding one line code.
python3 on Mac urllib3 == 1.23 requests == 2.19.1
so, I open the file “…/urllib3/response.py”, add
line = (len(line)>0 and line or "0")in 571, like below:This works fine with requests 2.10.0 so something regressed between then and 2.13.0. Also tested with 2.18.4 and the bug exists as above.
We temporarily repaired it in this way. I wonder if there will be any problem.
@KonMann you seem to have more time than me. Go for it.