pytest: get rid of "ImportMismatchError"

I encounter that error when running tests in a vagrant project. Vagrant maps your project read/writable into the box and pytest gets confused about that. Removing *.pyc in between runs when switching between guest and host resolves that.

Another case where this happens when running tests in a Docker container, where the source gets bind-mounted as an image (into something like /srv, /app or /src typically), and you had run the tests on the host already. In this case no automatic redirection would be possible.

I encounter this issue frequently. I do have multiple environments (host machine, docker, vm, etc.) and python versions (python 2 & 3), but this also happens when switching branches on a git repo to checkout patches to our main branch. For each environment, because I develop, I typically install the python package in development mode, which creates a .egg-info file, in addition to populating the folder tree with python bytecode files and __pycache__ folders. For some time, I also had pytest installed on my host system outside of the virtual environment, and that also caused the same kind of error.

I’ve internally codified the following cleanup procedure which I (potentially) run each time I switch branches:

$ hash -r
$ find . -name '*.py[odc]' -type f -delete
$ find . -name '__pycache__' -type d -delete
$ rm -Rf .pytest_cache
$ rm -rf *.egg-info .cache .eggs build dist
$ pip install -e .

The above seems to take care of the odd behavior.

in python3 you need to remove the pycache folders.

They also just contain .pyc files, so getting rid of the folders is not necessary - just the contents.

e.g. rm -r **/*.pyc

@hpk42 and @RonnyPfannschmidt thanks!

How about we add some way to force pytest to not raise the ImportMismatchError? This makes it possible to bypass this check on situations where we know for sure the error is a false positive, for example when running under tox. I’m thinking of something non-intrusive and simple to implement like PYTEST_IGNORE_IMPORT_MISMATCH=1. With it in pytest, tox could always set that environment variable behind the scenes automatically even if the user is not using pytest, which would solve the problem for all tox users.

While of course that’s not the most elegant solution and does not address the underlying issue, it has some advantages to it:

1. Very simple to implement so we can get this out very soon: tox sets an environment variable, pytest reads it and does not raise the error;
2. Basically no maintenance burden;
3. Backward compatible, for tox and pytest;
4. Can be kept around even if we implement a proper solution later, because as this discussion shows the solution probably isn’t easy and it might contain bugs itself.

This is not meant to solve the other user cases you described, but at least would solve the issue for a lot of tox users now, while we can discuss and refine a more long term solution. We might even decide to not document it at all, keep it as an “internal workaround” to at least alleviate tox users for now.

Just throwing this idea here to see what you guys think, don’t mean to disrupt/stop the current discussion on how to treat the problem properly.

I also find this error running pytest inside a docker container. The obestwalter’s tip works well for me, but i would like to add that in python3 you need to remove the __pycache__ folders.

@hpk42 proposal: ignoring the error altogether if the contents of the wanted and the imported file are the same (and adding a mention of install/reinstall/develop maybe)

@RonnyPfannschmidt Please read some of the above comments: it typically happens when running tests first withing Docker and then without - that is caused by the file paths typically being different inside the Docker container, regardless of the project layout. Also using symlinks appears to cause it (https://github.com/pytest-dev/pytest/issues/2042#issuecomment-423601601) in general. I think we should add support for py to ignore this in case some env variable is set. That is useful in general.

Just ran into this issue with trying to run pytests in a docker container. After removing the usual pyc suspects, found out pytest has it’s own cache, .pytest_cache that I had missed in .dockerignore / .gitignore.

Would patching this in py make sense?

https://github.com/pytest-dev/py/blob/bb4d0d0ca5c4b18a4b1276e2ccae3adf7f472bfa/py/_path/local.py#L685-L686

So that the line reads: if not issame and not os.getenv('PY_IGNORE_IMPORT_MISMATCH'):

This was on Ubuntu Trusty, I can’t paste the output as it was in a closed source repo.

I have the problem resolved now, but in my case, I worked around by setting (in tox.ini)

[pytest]
testpaths = src

Once that was set, pytest stopped seeing the .tox/ dir, and installed package there, causing the error.

I hold that the automatic redirection from source to install location is worth trying and could resolve this problem by default with pytest-4.0 (if not earlier – nobody can really have relied on an ImportMismatchError appearing).

py-1.7.0 has been released which supports using PY_IGNORE_IMPORTMISMATCH=1 to suppress this error. While this is not ideal, it is at least a escape hatch for cases where it is getting in the way. Thanks @blueyed for the patch! 🙇

they are at different location and different kind of plug-in have massive issues with that fact (and every time its happens, its a indication that the project has a broken folder structure)

Oh good point, might be worth trying to port pyimport. 👍

Hi @christianmlong, as a tox specific note: your tox.ini is equivalent to

[tox]
envlist = py36
skipsdist = True

[testenv]
usedevelop = True
extras = test
commands = pytest
deps = -rrequirements.txt

so you don’t need to emit any pip command and can control how it is installed purely by setting usedevelop.

see extras see py.test vs pytest

When working with docker, one handy option is to add .dockerignore, which will exclude all __pycache__ folders from being sent to the docker image context:

**/__pycache__

@scottschreckengaust currently doctest-modules is fundamentally incompatible with testing against a installed package due to module search in checkout vs module usage from site-packages

For anyone finding this page as a result of it being the first search result for a combination of the error message text and ‘tox’, there is a tox document with some suggested fixes. I hope that by putting this note here, others will be saved the frustrating day I’ve had trying to fix this issue with tox/pytest interaction and importable tests.

The section “installed-versus-checkout version” here: http://tox.readthedocs.io/en/latest/example/pytest.html#known-issues-and-limitations has a good explanation and solutions for this error.

(All that is not to say I wouldn’t like pytest to take care of this for me 😄)

Still a good intermediate step likely to allow for ignoring it using an environment variable?!

IMHO yes, that solution would probably be enough for most use cases and very simple to implement.

Affected, too. Would be great to have an environment variable to bypass the checks.

Still a good intermediate step likely to allow for ignoring it using an environment variable?!

With pytest / docker, I ran into this issue, but solved it by deleting .pyc files.

On Unix, an easy solution is to run the following command: find . -name “*.pyc” -delete

Hope this helps

@letmaik I would be very surprised if a complete removal of all caches and python cruft would not solve the problem. If that really is the case, it would be helpful if you could provide a minimal reproducer or a link to the repository if it is publc and the exact steps to reproduce the problem.

I can confirm that obestwalter’s tip works well for Docker also.

I am also having this issue when using the Windows Subsystem for Linux and trying to run my tests there. Apparently the linux/windows paths get mixed up.

I am not sure if this is a separate issue as I am somewhat surprised how the windows path even gets in there in the first place. Here is the traceback (with shortened paths):

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/_pytest/config.py", line 325, in _getconftestmodules
    return self._path2confmods[path]
KeyError: local('/mnt/c/.../tests')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/_pytest/config.py", line 356, in _importconftest
    return self._conftestpath2mod[conftestpath]
KeyError: local('/mnt/c/.../tests/conftest.py')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/_pytest/config.py", line 362, in _importconftest
    mod = conftestpath.pyimport()
  File "/usr/local/lib/python3.5/dist-packages/py/_path/local.py", line 668, in pyimport
    raise self.ImportMismatchError(modname, modfile, self)
py._path.local.LocalPath.ImportMismatchError: ('tests.conftest', 'C:\\...\\tests\\conftest.py', local('/mnt/c/.../tests/conftest.py'))
ERROR: could not load /mnt/c/.../tests/conftest.py

I thought you might ask that, and the answer is essentially no. Or rather, on Python 3.3+, any directory on sys.path is always the root of a package:

$ mkdir foo
$ python -c "import foo; print(foo.__path__)"
_NamespacePath(['/Users/jaraco/foo'])

Probably the packaging infrastructure should expose the “namespace packages” declared for a given distribution, such that you could enumerate all installed packages and determine the namespace packages they declare (and from there resolve the paths pertinent to those packages), but in my initial investigation, I don’t see that metadata exposed for installed packages.