HTTPretty: Duplicate requests in latest_requests() if there are chunks

Was fine with 1.0.5, does duplicates in 1.1.0.

The change: https://github.com/gabrielfalcao/HTTPretty/commit/b6161cebfdaf5f68ef49526e51b460b8e0c0c6c2#r50812634

-            cls.latest_requests[-1] = request
+            pos = cls.latest_requests.index(request)
+            cls.latest_requests[pos] = request

httpretty.historify_request() is called multiple times for the same request.

From here:

  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.6/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1081, in _send_output
    self.send(chunk)
  File "/usr/lib/python3.6/http/client.py", line 1002, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.6/dist-packages/httpretty/core.py", line 683, in sendall

And here:

  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.6/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1042, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 1002, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.6/dist-packages/httpretty/core.py", line 699, in sendall

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 14
  • Comments: 24 (9 by maintainers)

Commits related to this issue

Most upvoted comments

+1 for a 1.1.5 release

I am dealing with the exact same issue. Looks like there was a fix merged into the master. As @ThiefMaster was pointing out, @gabrielfalcao would it be possible to deploy a new release?

Thank you for creating a tool that helps improve automated tests! 🤗

I recently encountered this issue and wanted to add more info. In my testing, this applies to any requests (POST, PUT, DELETE, and/or PATCH) when they contain a request body. Below is a test file that demonstrates this behavior. (I did not include tests with request bodies for HTTP methods which do not allow request bodies (GET, HEAD, CONNECT, and OPTIONS), however these also demonstrate the behavior when a request body is provided.)

import httpretty
import urllib.request as request


class TestHTTpretty:
    @httpretty.activate(verbose=True, allow_net_connect=False)
    def run(self, method, data=None):
        url = 'https://example.com/foo'
        httpretty.register_uri(method=method, uri=url)
        req = request.Request(url=url, method=method, data=data)
        request.urlopen(req)
        assert len(httpretty.latest_requests()) == 1

    # the first eight tests with empty request bodies succeed
    def test_get(self):
        self.run(httpretty.GET)

    def test_head(self):
        self.run(httpretty.HEAD)

    def test_connect(self):
        self.run(httpretty.CONNECT)

    def test_options(self):
        self.run(httpretty.OPTIONS)

    def test_post_empty(self):
        self.run(httpretty.POST)

    def test_put_empty(self):
        self.run(httpretty.PUT)

    def test_delete_empty(self):
        self.run(httpretty.DELETE)

    def test_patch_empty(self):
        self.run(httpretty.PATCH)

    # the following four tests with populated request bodies fail
    def test_post_with_data(self):
        self.run(httpretty.POST, b'lorem ipsum')

    def test_put_with_data(self):
        self.run(httpretty.PUT, b'lorem ipsum')

    def test_delete_with_data(self):
        self.run(httpretty.DELETE, b'lorem ipsum')

    def test_patch_with_data(self):
        self.run(httpretty.PATCH, b'lorem ipsum')

Output:

============================ short test summary info ============================
FAILED test/httpretty_test.py::TestHTTpretty::test_post_with_data - assert 2 == 1
FAILED test/httpretty_test.py::TestHTTpretty::test_put_with_data - assert 2 == 1
FAILED test/httpretty_test.py::TestHTTpretty::test_delete_with_data - assert 2 == 1
FAILED test/httpretty_test.py::TestHTTpretty::test_patch_with_data - assert 2 == 1

Hi @gabrielfalcao,

This is a friendly reminder; I hope you’ll find some time to make a new release 😃

Thank you

Hi folks, I presently find myself somewhat able to work on this issue. I’ll do what I can to make a release. Thanks for your patience so far.

@gabrielfalcao I can confirm that it only appears to impact POST requests as @tim-schilling suggested:

import httpretty
import requests

httpretty.enable()

httpretty.register_uri(method=httpretty.POST, uri="https://example.com/foo")
requests.post("https://example.com/foo", data="bar")
len(httpretty.latest_requests())
# returns 2

httpretty.reset()

httpretty.register_uri(method=httpretty.GET, uri="https://example.com/foo")
requests.get("https://example.com/foo")
len(httpretty.latest_requests())
# returns 1

Thanks @daveygit2050 I’ll incorporate this snippet into a new test case to reproduce the issue as soon as I can manage. Unfortunately I might not be able to attend to this until next week but I’ll keep yall posted.

These duplicate requests broke my CI. Pinning httpretty<1.1 is a workaround but a fix would be appreciated.

Problem exists in httpretty==1.1.4 httpretty==1.0.5 works as a workaround

@gabrielfalcao thanks for your great effort, it is appreciated!

Salutations all,

my macbook got damaged and I’m quite a bit limited at present due to the time taken for the new parts to arrive. I plan on applying the patch and resolve the situation as soon as the universe collaborates. Please hang in there and I’ll report back as soon as humanely possible.

Kind regards, Gabriel

@gabrielfalcao Can we please get an 1.1.5 release containing this fix?

@gabrielfalcao Same result for me. Thanks for getting on this quickly. I’ll try to get an MWE together at the end of my sprint.