pytest: Testing of `pytest` fails with pytest plugins installed

I’m packaging pytest for OpenIndiana. When I run tests for pytest I notice that testing fails if there are some pytest plugins installed, namely pytest-black, pytest-mypy, and pytest-cov. The failure is this:

__________________ TestConfigFromdictargs.test_basic_behavior __________________
 
self = <test_config.TestConfigFromdictargs object at 0x7fffab105a60>
_sys_snapshot = None 

    def test_basic_behavior(self, _sys_snapshot) -> None:
        option_dict = {"verbose": 444, "foo": "bar", "capture": "no"}
        args = ["a", "b"]
    
        config = Config.fromdictargs(option_dict, args)
        with pytest.raises(AssertionError):
            config.parse(["should refuse to parse again"])
        assert config.option.verbose == 444
        assert config.option.foo == "bar"
        assert config.option.capture == "no"
>       assert config.args == args
E       AssertionError: assert ['a', 'b'] == ['a', 'b', '-...ypy', '--cov']
E         Right contains 3 more items, first extra item: '--black'
E         Full diff:
E         - ['a', 'b', '--black', '--mypy', '--cov']
E         + ['a', 'b']

testing/test_config.py:895: AssertionError

When I modify the test using the following patch:

--- pytest-7.4.1/testing/test_config.py.orig
+++ pytest-7.4.1/testing/test_config.py
@@ -886,7 +886,7 @@
         option_dict = {"verbose": 444, "foo": "bar", "capture": "no"}
         args = ["a", "b"]
 
-        config = Config.fromdictargs(option_dict, args)
+        config = Config.fromdictargs(option_dict, ["-p", "no:foobar"] + args)
         with pytest.raises(AssertionError):
             config.parse(["should refuse to parse again"])
         assert config.option.verbose == 444

then the test pass.

About this issue

Original URL
State: closed
Created 10 months ago
Comments: 15 (8 by maintainers)

Most upvoted comments

I got curious on your mystery why doing ["-p", "no:foobar"] + args fixed the issue. It’s not actually about the -p argument, doing [] + args works just as well.

The reason why the test failed is that the Config.fromdictargs(option_dict, args) call mutates the passed in args (because that’s what pytest-enabler does). And then the comparison fails not because config.args changed, but because the expected value (args) did change:

-> assert config.args == args
(Pdb) p config.args
['a', 'b']
(Pdb) p args
['a', 'b', '--black', '--mypy', '--cov']

By passing in [...] + args, you effectively copy args, and thus the expected value doesn’t get mutated by pytest-enabler, but an independent list (which is not reused) does.

The-Compiler on Sep 6, 2023

For what it’s worth, I cannot reproduce:

$ python3 -m venv .venv
$ .venv/bin/pip install -q pytest-black pytest-mypy pytest-cov
$ .venv/bin/pytest testing/test_config.py::TestConfigFromdictargs::test_basic_behavior
============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.1, pluggy-1.3.0
rootdir: /home/florian/proj/pytest
configfile: pyproject.toml
plugins: cov-4.1.0, black-0.3.12, mypy-0.10.3
collected 1 item                                                               

testing/test_config.py .                                                 [100%]

============================== 1 passed in 0.15s ===============================

It sounds like you have something else possibly going on in your environment adding those arguments, because I don’t see how simply installing those plugins would add command line arguments to the default configuration.

The-Compiler on Sep 6, 2023