pytest: `'store_true'` does not work correctly in parser.addoption

One more from https://github.com/conda-forge/staged-recipes/pull/16888, where (after patching in workarounds for #9300 & #9301), the test suite failed with

==================================== ERRORS ====================================
________________ ERROR at setup of test_serialization[cpu-gelu] ________________

self = <_pytest.config.Config object at 0x7fc1975d41c0>
name = 'skip_custom_ops', default = <NOTSET>, skip = False

    def getoption(self, name: str, default=notset, skip: bool = False):
        """Return command line option value.
    
        :param name: Name of the option.  You may also specify
            the literal ``--OPT`` option instead of the "dest" option name.
        :param default: Default value if no option of that name exists.
        :param skip: If True, raise pytest.skip if option does not exists
            or has a None value.
        """
        name = self._opt2dest.get(name, name)
        try:
>           val = getattr(self.option, name)
E           AttributeError: 'Namespace' object has no attribute 'skip_custom_ops'

../[...]/_pytest/config/__init__.py:1463: AttributeError

The above exception was the direct cause of the following exception:

request = <SubRequest 'set_global_variables' for <Function test_serialization[cpu-gelu]>>

    @pytest.fixture(scope="session", autouse=True)
    def set_global_variables(request):
>       if request.config.getoption("--skip-custom-ops"):
E       ValueError: no option named 'skip_custom_ops

So from that, it looks like something is going wrong with skip_custom_ops. Chasing around the upstream repo I find that this option is added here as follows:

def pytest_addoption(parser):
    parser.addoption(
        "--skip-custom-ops",
        action="store_true",
        help="When a custom op is being loaded in a test, skip this test.",
    )

The pytest docs for parser.addoption say that this works like argparse, where the linked docs say:

'store_true' and 'store_false' - These are special cases of 'store_const' used for storing the values True and False respectively. In addition, they create default values of False and True respectively.

What caught my eye in the stack trace was the default = <NOTSET>, which is clearly at odds with that documentation. To verify this hypothesis, I added the following patch

diff --git a/tensorflow_addons/utils/test_utils.py b/tensorflow_addons/utils/test_utils.py
index 9d1dec0..0efd707 100644
--- a/tensorflow_addons/utils/test_utils.py
+++ b/tensorflow_addons/utils/test_utils.py
@@ -134,7 +134,8 @@ def set_seeds():
 def pytest_addoption(parser):
     parser.addoption(
         "--skip-custom-ops",
-        action="store_true",
+        action="store",
+        default=True,
         help="When a custom op is being loaded in a test, skip this test.",
     )

and voilà, it works.

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 15 (8 by maintainers)

Most upvoted comments

Hi looking through this issue I was able to reproduce the error @h-vetinari ran into with the following python script:

from _pytest.config import get_config

cfg = get_config()

# This does not work
cfg._parser.addoption(
    "--deadbeef",
    action="store_true"
)
cfg.getoption("--deadbeef") # value error thrown here

while changing the arguments passed to addoptions to @h-vetinari workaround fixes things:

from _pytest.config import get_config

cfg = get_config()

# This works
cfg._parser.addoption(
    "--livebeef",
    action="store",
    default=True
)
cfg.getoption("--livebeef")

I’m wondering if the reason @yuvalshi0 wasn’t able to reproduce this error is because pytester.parseconfig or pytester.makeconftest is patching the bug. I want to look further into this issue and work on a fix for it, just don’t want to step on anyone’s toes if they’re already working on a fix.

@kevinsantana11 it may be helpfull to put the conftest into a subdirectory

i suspect the issue may be related to conftest discovery and initial conftest loading

for ensuring the rootdir, adding a pytest.ini or a setup.cfg may help

@RonnyPfannschmidt thanks for the feedback, will work on creating a proper test for reproducing the error instead of the hacky snippet I used. Creating a proper reproduction of the error should also help in finding/developing a fix.