vcrpy: CannotOverwriteExistingCassetteException with no matchers failed

I’m trying to remove access tokens from the response body of some http transactions that are being recorded so that I can check the sanitized cassettes into source control so I’ve added a before_record_request to the vcr.

import json

def scrub_access_token(response):
    body = response.get("body", {}).get("string")
    if body is None:
        return response
    try:
        body = json.loads(body)
    except Exception:
        return response

    if "access_token" not in body:
        return response

    body["access_token"] = "REDACTED"
    new_body = json.dumps(body)
    response["body"]["string"] = new_body
    response["headers"]["Content-Length"] = len(new_body)
    return response

@pytest.mark.vcr(before_record_response=scrub_access_token)
def test_foo:
    ...

It runs successfully the first run and correctly removes the field from the body, but then on subsequent runs, I get an error when requesting that request.

E               vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('/home/bmatase/cassettes/test_become_active.yaml') in your current record mode ('none').
E               No match for the request (<Request (GET) http://169.254.169.254/metadata/identity/oauth2/token?resource=https%3A%2F%2Fmanagement.core.windows.net%2F&api-version=2018-02-01>) was found.
E               Found 1 similar requests with 0 different matcher(s) :
E               
E               1 - (<Request (GET) http://169.254.169.254/metadata/identity/oauth2/token?resource=https%3A%2F%2Fmanagement.core.windows.net%2F&api-version=2018-02-01>).
E               Matchers succeeded : ['method', 'scheme', 'host', 'port', 'path', 'query']
E               Matchers failed : 

I don’t understand why the two requests aren’t matching if all of the matchers matched. It seems odd that the requests aren’t matching because I’m modifying the reponse. I originally thought that it was because of the content length not matching, but it still occurs after that addition.

I don’t think this is a duplicate of #516 since there aren’t two identical requests in the error message.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 26
  • Comments: 19

Most upvoted comments

Indeed, allow_playback_repeats=True solves the issue. I guess we could just modify the error message to say that there is a matching request, but it has already been used and that this option should be provided to allow reusing it.

Hey folks!

Found this issue after having spent half an hour trying to figure out the mysterious case with “no matchers failed”. Turns out that indeed there was an unwanted repeat-request in my code.

Having a more helpful failure message would have saved me a bit of head-scratching. Would it make sense to change the message that prints out the fact that the request was repeated and even suggests turning on allow_playback_repeats?

Is it possible that this is a case where there is incomplete feedback in the case of reuse of a request inside a cassette? I circumvented my problem by enabling allow_playback_repeats. Given that, my guess is that perhaps I’m inadvertently requesting the same URL twice even though it was only requested once during recording. It’s possible that some additional feedback might help users to better understand this functionality.

This PR combined with the following test/conftest.py file resolved the issue for me:

from vcr.persisters.deduplicated_filesystem import DeduplicatedFilesystemPersister

def pytest_recording_configure(config, vcr):
  vcr.register_persister(DeduplicatedFilesystemPersister)

+1, I am having this issue as well, but I am using the pytest-vcr, I can’t pass allow_playback_repeats through to vcrpy. Will watch this thread.

Might not be helpful to you anymore but using pytest-vcr, you can pass allow_playback_repeats like this

@pytest.mark.vcr(allow_playback_repeats=True)
def test_func() -> None:
    ...

I am having the same issue and it seems to be intermittent. I have also tried setting allow_playback_repeats to true, which does not work for me, and setting RecordMode to NEW_EPISODES (which results in the test attempting to make the API call again, rather than using the cassette).

My workaround is to mock the API call that needs to be called twice in the test and was creating the issue.

For pytest-vcr users, a conftest.py can contain some setup fixtures

The new_episodes record-mode solved this issue for me.

For example:

@pytest.fixture(scope="module")
def vcr_config():
    # For any live API requests, do not record the API-token
    # See https://vcrpy.readthedocs.io/en/latest/advanced.html
    return {
        "record_mode": "new_episodes",
        "filter_query_parameters": [("key", "APIKeyXXX")],
        "filter_headers": [("x-api-key", "X-API-KEY-XXX")]
    }


@pytest.fixture(scope='module')
def vcr(vcr):
    vcr.register_matcher('my_matcher', my_matcher)
    vcr.match_on = ['my_matcher']  # This can also go into vcr_config or marker kwargs
    return vcr

👋🏾 I’m coming back to Pythonland after being out for a number of years and still trying to get my bearings. I also came across this with code very similar to the original poster’s. I found this to actually work for me after modifying the response string. I have not looked into this enough to be able to tell you why though 😅 .

response['body']['string'] = bytes(json.dumps(json_body), 'utf8')

I tried to reproduce this error. Here are the steps I performed.

  1. I run the following test
import vcr
import requests

@vcr.use_cassette()
def test_iana():
    requests.get('http://www.iana.org/domains/reserved')
    requests.get('http://www.iana.org/domains/reserved')
  1. In the casssette file test_iana, I deleted one request.

  2. I then run the test again, here is the error message I got:

E               vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('/home/mossaka/developer/azureml-v2/sdk-cli-v2/jiazho_playground/test_iana') in your current record mode (<RecordMode.ONCE: 'once'>).
E               No match for the request (<Request (GET) http://www.iana.org/domains/reserved>) was found.
E               Found 1 similar requests with 0 different matcher(s) :
E               
E               1 - (<Request (GET) http://www.iana.org/domains/reserved>).
E               Matchers succeeded : ['method', 'scheme', 'host', 'port', 'path', 'query']
E               Matchers failed :