pytest: Regression in 7.1 ? Windows CI started to fail with "assert mod not in mods" since 7.1
Now sorry I know this is not a minimal reproducer, but I just want to log this here, and will investigate later:
You can see one Failure here
ERROR collecting test session
C:\hostedtoolcache\windows\Python\3.9.10\x64\lib\site-packages\_pytest\runner.py:338: in from_call
result: Optional[TResult] = func()
C:\hostedtoolcache\windows\Python\3.9.10\x64\lib\site-packages\_pytest\runner.py:369: in <lambda>
call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
C:\hostedtoolcache\windows\Python\3.9.10\x64\lib\site-packages\_pytest\main.py:7[23](https://github.com/ipython/ipython/runs/5535021197?check_suite_focus=true#step:8:23): in collect
for x in self._collectfile(pkginit):
C:\hostedtoolcache\windows\Python\3.9.10\x64\lib\site-packages\_pytest\main.py:575: in _collectfile
ihook = self.gethookproxy(fspath)
C:\hostedtoolcache\windows\Python\3.9.10\x64\lib\site-packages\_pytest\main.py:539: in gethookproxy
my_conftestmodules = pm._getconftestmodules(
C:\hostedtoolcache\windows\Python\3.9.10\x64\lib\site-packages\_pytest\config\__init__.py:579: in _getconftestmodules
mod = self._importconftest(conftestpath, importmode, rootpath)
C:\hostedtoolcache\windows\Python\3.9.10\x64\lib\site-packages\_pytest\config\__init__.py:6[24](https://github.com/ipython/ipython/runs/5535021197?check_suite_focus=true#step:8:24): in _importconftest
assert mod not in mods
E AssertionError
Now I’m not sure what this does, but my quick understanding is that it checks whether conftest.py itself is not in the list of collected modules ? We might definitely do something wrong on IPython side, but it’s strange as other non-windows CI are passing and regardless of whether this is something I did wrong, can the assert get an explanation messages as second argument ?
Downgrading to <7.1 fixes the issue.
Again I’ll try to get a MVP/small example, but it may take me some time, and I guessed maybe one of you will immediately know why this is happening.
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 3
- Comments: 81 (47 by maintainers)
Commits related to this issue
- Avoid regression on pytest 7.1 See https://github.com/pytest-dev/pytest/issues/9765 — committed to compas-dev/compas by gonzalocasas 2 years ago
- CI: use pytest<7.1 on Windows - https://github.com/pytest-dev/pytest/issues/9765 [skip ci] — committed to cctbx/cctbx by bkpoon 2 years ago
- Pin pytest to avoid assertion failure 'assert mod not in mods' when run on Windows. See https://github.com/pytest-dev/pytest/issues/9765 — committed to martinburchell/crate by martinburchell 2 years ago
- CI: use pytest=7.0 for Windows tests - https://github.com/pytest-dev/pytest/issues/9765 [skip ci] — committed to bkpoon/cctbx_project by bkpoon 2 years ago
- CI: use pytest=7.0 for Windows tests - https://github.com/pytest-dev/pytest/issues/9765 [skip ci] — committed to bkpoon/cctbx_project by bkpoon 2 years ago
- Normalize conftest paths Refs #9765. While this is probably not needed for fixing the `assert mod not in mods` bug (previous commit takes care of it), I believe this is the right thing to do to avoid... — committed to fcharras/pytest by bluetech 6 months ago
@bluetech any news on this? Anything newer than pytest 7.0.1 is still broken for me on linux, including the latest 7.2.1. I’ve got one conftest.py in my root, and several smaller conftest.py in subfolders for different django apps. This works fine in 7.0.1 and before, but fails to run at all in any newer version, with the following output:
@pllim this seems very similar to the Scipy one,
os.path.normcaseissue, capitalLLib(inmod.__file__) vs smallllib(inconftestpath):I think this could be fixed by https://github.com/pytest-dev/pytest/pull/11708. Hopefully we manage to push the PR forward tomorrow.
@pllim not sure what is going on, ideally you would cherry-pick my simple debug commit on top of whichever version you were seeing the
assert mod not in modserror before.@pllim can you try with (cherry-picked the commit but using the 0.8.x branch):
and post the output?
I guess @tylerjereddy you can do this as well?
While we still don’t have a proper reproduction for all cases, I believe we have a pretty good grasp of the issue and how to fix it (see https://github.com/pytest-dev/pytest/issues/9765#issuecomment-1869103766). Since @fcharras presumably hasn’t had time to update PR #11708, I’ll try to update it myself in the next few days so that we can backport it to pytest 8.0 release.
@pllim is it easily doable in your CI to install pytest from my branch:
and post the output?
I added a quick and dirty commit that should add a bit more information to troubleshoot.
@fcharras I’m using a regular ext4 case-sensitive FS on linux. These are the variables you asked for:
conftestpath=PosixPath('/srv/vidicenter/conftest.py')mod.__file__='/srv/vidicenter/../vidicenter/conftest.py'@fcharras @bluetech I can confirm that this solves the problem for me on Linux!
After thorough investigation with @lesteve slowly comparing the loading steps from both two environments on windows, one of those showcasing the issue, the other being sane, we found that for some reason the paths of the conftest files are sometimes small cased when used as keys in the
name2plugindictionary, but then the same small-casing is not applied at inspection time. This leads to the double loading of plugins, and the subsequent crash.https://github.com/pytest-dev/pytest/pull/11708 fixes that.
Could other users confirm that the fix work for them too ?
I’ve got the same error, but on linux! Can’t run any test on 7.1.x, reverting back to 7.0.x fixes the issue: