pytest: 6.2.4: pytest is failing when setuptools produces DSO module

It is quite regular cases and there are many such modules which are producucing dururing python setup.py build command DSO loadable module for example using cython. As I wrote it is quite regular that such modules are failing on testing using pytest because pytest cannot locate DSO module. Such example could be yarl (https://github.com/aio-libs/yarl/) but there are many more of them To reproduce could be used below script:

wget -O - https://github.com/aio-libs/yarl//archive/v1.6.3/yarl-1.6.3.tar.gz| tar xz
cd yarl-1.6.3
python3 -m cython -3 -o yarl/_quoting_c.c yarl/_quoting_c.pyx -I yarl
python3 setup.py  build
python3 -Bm pytest -ra

In my case above fails with:

 python3 -Bm pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
Using --randomly-seed=3950045766
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/tkloczko/rpmbuild/BUILD/yarl-1.6.3/yarl-1.6.3/.hypothesis/examples')
rootdir: /home/tkloczko/rpmbuild/BUILD/yarl-1.6.3/yarl-1.6.3, configfile: setup.cfg, testpaths: tests/
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, cases-3.4.6, case-1.5.3, isort-1.3.0, aspectlib-1.5.2, asyncio-0.15.1, toolbox-0.5, xprocess-0.17.1, aiohttp-0.3.0, checkdocs-2.7.0, mock-3.6.1, rerunfailures-9.1.1, requests-mock-1.9.3, Faker-8.4.0, cov-2.12.1, randomly-3.8.0, pyfakefs-4.5.0, hypothesis-6.13.14
collected 498 items / 1 error / 497 selected
Coverage.py warning: No data was collected. (no-data-collected)

================================================================================== ERRORS ==================================================================================
__________________________________________________________________ ERROR collecting tests/test_quoting.py __________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/yarl-1.6.3/yarl-1.6.3/tests/test_quoting.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_quoting.py:9: in <module>
    from yarl._quoting_c import _Quoter as _CQuoter, _Unquoter as _CUnquoter
E   ModuleNotFoundError: No module named 'yarl._quoting_c'

----------- coverage: platform linux, python 3.8.9-final-0 -----------
Name                  Stmts   Miss  Cover
-----------------------------------------
yarl/__init__.py          3      3     0%
yarl/_quoting.py         10     10     0%
yarl/_quoting_py.py     155    155     0%
yarl/_url.py            569    569     0%
-----------------------------------------
TOTAL                   737    737     0%

========================================================================= short test summary info ==========================================================================
ERROR tests/test_quoting.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================= 1 error in 1.34s =============================================================================

After build it is produces build/lib.<os-arch>-<pyver>/yarl/_quoting_c.cpython-*.so

[tkloczko@barrel yarl-1.6.3]$ ls -l build/lib.linux-x86_64-3.8/yarl/_quoting_c.cpython-38-x86_64-linux-gnu.so
-rwxrwxr-x 1 tkloczko tkloczko 475824 Jun  5 07:13 build/lib.linux-x86_64-3.8/yarl/_quoting_c.cpython-38-x86_64-linux-gnu.so

Something is missing in such cases or it some bug in pytest? Because it is very common case I suppose that it could be something wrong with pytest (?)

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 37 (22 by maintainers)

Most upvoted comments

I just realized the error

Its a Pytest invocation error

Invocation as module changes pythonpath

That needs to be accounted for, for example by using a src folder

Yarl does not

A potential workaround is to invoke the tests in the test subdirectory

The general suggestion is that if you want to to test either install, build inplace or do a editableinstall

Thereis no pytest to setuptools integration a spart of pytest