tools: pytest version 7 throwing PytestRemovedIn8Warning because of py.path.local use in pytest_workflow

Yes, I tried that on Gitpod.

The --git-aware flag in

PROFILE=docker pytest --tag fastqc --symlink --keep-workflow-wd --git-aware

solves the UserWarning: .git dir detected.

No. Do you know where to address it?

I don’t exactly, but the message sounds to me like a --git-aware flag should be added to the pytest call (wherever that is done).

@mahesh-panchal This “failure” will manifest independent of Gitpod, so it might be good to title this issue something not Gitpod specific like “pytest version 7 throwing PytestRemovedIn8Warning because of py.path.local use in pytest_workflow”.

tl;dr: Unsurprisingly there’s still use of the old Python path handling. It is originating from pytest workflow 1.6 and will be resolved in the next release.

How to find out: Here’s a more useful pytest output (note the -W error to make warnings behave like errors producing tracebacks and the --tb=long to see the complete traceback):

gitpod /workspace/modules (master) $ PROFILE=docker pytest --tag fastqc --symlink --keep-workflow-wd --tb=long -W error -v
================================================================================================================================================ test session starts =================================================================================================================================================
platform linux -- Python 3.9.7, pytest-7.0.1, pluggy-1.0.0 -- /opt/conda/bin/python
cachedir: .pytest_cache
rootdir: /workspace/modules
plugins: workflow-1.6.0
collecting ... 
collected 0 items / 1 error                                                                                                                                                                                                                                                                                          

======================================================================================================================================================= ERRORS =======================================================================================================================================================
___________________________________________________________________________________________________________________________________________ ERROR collecting test session ____________________________________________________________________________________________________________________________________________

self = <_HookCaller 'pytest_collect_file'>, args = (), kwargs = {'file_path': PosixPath('/workspace/modules/tests/modules/abacas/test.yml'), 'parent': <Session modules exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>, 'path': local('/workspace/modules/tests/modules/abacas/test.yml')}
argname = 'parent', firstresult = False

    def __call__(self, *args, **kwargs):
        if args:
            raise TypeError("hook calling supports only keyword arguments")
        assert not self.is_historic()
    
        # This is written to avoid expensive operations when not needed.
        if self.spec:
            for argname in self.spec.argnames:
                if argname not in kwargs:
                    notincall = tuple(set(self.spec.argnames) - kwargs.keys())
                    warnings.warn(
                        "Argument(s) {} which are declared in the hookspec "
                        "can not be found in this hook call".format(notincall),
                        stacklevel=2,
                    )
                    break
    
            firstresult = self.spec.opts.get("firstresult")
        else:
            firstresult = False
    
>       return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)

/opt/conda/lib/python3.9/site-packages/pluggy/_hooks.py:265: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_pytest.config.PytestPluginManager object at 0x7fee59bcac70>, hook_name = 'pytest_collect_file'
methods = [<HookImpl plugin_name='python', plugin=<module '_pytest.python' from '/opt/conda/lib/python3.9/site-packages/_pytest/...ow', plugin=<module 'pytest_workflow.plugin' from '/opt/conda/lib/python3.9/site-packages/pytest_workflow/plugin.py'>>]
kwargs = {'file_path': PosixPath('/workspace/modules/tests/modules/abacas/test.yml'), 'parent': <Session modules exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>, 'path': local('/workspace/modules/tests/modules/abacas/test.yml')}, firstresult = False

    def _hookexec(self, hook_name, methods, kwargs, firstresult):
        # called from all hookcaller instances.
        # enable_tracing will set its own wrapping function at self._inner_hookexec
>       return self._inner_hookexec(hook_name, methods, kwargs, firstresult)

/opt/conda/lib/python3.9/site-packages/pluggy/_manager.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

path = local('/workspace/modules/tests/modules/abacas/test.yml'), parent = <Session modules exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>

    def pytest_collect_file(path, parent):
        """Collection hook
        This collects the yaml files that start with "test" and end with
        .yaml or .yml"""
        if path.ext in [".yml", ".yaml"] and path.basename.startswith("test"):
>           return YamlFile.from_parent(parent, fspath=path)

/opt/conda/lib/python3.9/site-packages/pytest_workflow/plugin.py:122: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'pytest_workflow.plugin.YamlFile'>, parent = <Session modules exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>, fspath = local('/workspace/modules/tests/modules/abacas/test.yml'), path = None, kw = {}

    @classmethod
    def from_parent(
        cls,
        parent,
        *,
        fspath: Optional[LEGACY_PATH] = None,
        path: Optional[Path] = None,
        **kw,
    ):
        """The public constructor."""
>       return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

/opt/conda/lib/python3.9/site-packages/_pytest/nodes.py:633: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'pytest_workflow.plugin.YamlFile'>, parent = <Session modules exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>, kw = {'fspath': local('/workspace/modules/tests/modules/abacas/test.yml'), 'path': None}

    @classmethod
    def from_parent(cls, parent: "Node", **kw):
        """Public constructor for Nodes.
    
        This indirection got introduced in order to enable removing
        the fragile logic from the node constructors.
    
        Subclasses can use ``super().from_parent(...)`` when overriding the
        construction.
    
        :param parent: The parent node of this Node.
        """
        if "config" in kw:
            raise TypeError("config is not a valid argument for from_parent")
        if "session" in kw:
            raise TypeError("session is not a valid argument for from_parent")
>       return cls._create(parent=parent, **kw)

/opt/conda/lib/python3.9/site-packages/_pytest/nodes.py:264: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <class 'pytest_workflow.plugin.YamlFile'>, k = (), kw = {'fspath': local('/workspace/modules/tests/modules/abacas/test.yml'), 'parent': <Session modules exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>, 'path': None}

    def _create(self, *k, **kw):
        try:
>           return super().__call__(*k, **kw)

/opt/conda/lib/python3.9/site-packages/_pytest/nodes.py:140: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <YamlFile None>, fspath = local('/workspace/modules/tests/modules/abacas/test.yml'), path_or_parent = None, path = None, name = None, parent = <Session modules exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=0>, config = None, session = None, nodeid = None

    def __init__(
        self,
        fspath: Optional[LEGACY_PATH] = None,
        path_or_parent: Optional[Union[Path, Node]] = None,
        path: Optional[Path] = None,
        name: Optional[str] = None,
        parent: Optional[Node] = None,
        config: Optional[Config] = None,
        session: Optional["Session"] = None,
        nodeid: Optional[str] = None,
    ) -> None:
        if path_or_parent:
            if isinstance(path_or_parent, Node):
                assert parent is None
                parent = cast(FSCollector, path_or_parent)
            elif isinstance(path_or_parent, Path):
                assert path is None
                path = path_or_parent
    
>       path = _imply_path(type(self), path, fspath=fspath)

/opt/conda/lib/python3.9/site-packages/_pytest/nodes.py:588: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

node_type = <class 'pytest_workflow.plugin.YamlFile'>, path = None, fspath = local('/workspace/modules/tests/modules/abacas/test.yml')

    def _imply_path(
        node_type: Type["Node"],
        path: Optional[Path],
        fspath: Optional[LEGACY_PATH],
    ) -> Path:
        if fspath is not None:
>           warnings.warn(
                NODE_CTOR_FSPATH_ARG.format(
                    node_type_name=node_type.__name__,
                ),
                stacklevel=3,
            )
E           pytest.PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
E           See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path

/opt/conda/lib/python3.9/site-packages/_pytest/nodes.py:110: PytestRemovedIn8Warning
============================================================================================================================================== short test summary info ===============================================================================================================================================
ERROR  - pytest.PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================================================================================== 1 error in 0.41s ==================================================================================================================================================
gitpod /workspace/modules (master) $ 

So the problem is in pytest wokflow which uses the fspath instead of path argument here YamlFile.from_parent(parent, fspath=path) in pytest_workflow.plugin.py on line 122. This has been fixed in the pytest workflow code - see this blame from this commit.

This issue should be resolved once pytest workflow 1.7 is released and used in nextflow/Gitpod.

Have a nice day 😃

PS: not closing this issue just now, because the solution has not manifested here in nf-core, yet.

On the Gitpod: https://gitpod.io/#https://github.com/nf-core/modules/, and then open a terminal from the “Hamburger” icon (three dashes) in the top left.

Alternatively, locally, you can do:

docker run --rm -it nfcore/gitpod:latest bash
sudo mkdir /workspace
sudo chown gitpod:gitpod /workspace
cd /workspace
git clone https://github.com/nf-core/modules.git
cd modules

And then to run the fastqc pytest, do:

PROFILE=docker pytest --tag fastqc --symlink --keep-workflow-wd