bazel: Python Coverage does not work
I thought that there was an existing issue for this, but I can’t find it.
I was able to successfully collect coverage for python code with a modified version of coverage.py:
- Use a Bazel binary that include b01c85962d88661ec9f6c6704c47d8ce67ca4d2a
- Check out the modified coverage.py from https://github.com/ulfjack/coveragepy/tree/lcov-support (this is based on the patch at https://github.com/nedbat/coveragepy/pull/863 with a small change to make it work with Bazel)
- Run bazel:
bazel coverage --test_env=PYTHON_COVERAGE=/path/to/coveragepy/coverage/__main__.py //python:lib_test
Test coverage should be written to bazel-testlogs/python/lib_test/coverage.dat in lcov format.
About this issue
- Original URL
- State: open
- Created 7 years ago
- Reactions: 26
- Comments: 21 (7 by maintainers)
@chickenandpork @ulfjack If you use a pytest_runner to run your pytest tests, it’s a convenient place to put the lconv conversion code 😉 That way you can still use standard tools (and not need to use the fork or special shell scripts)
The following requires
pytest-covandcoverage-lconvFYI, I created https://github.com/bazelbuild/rules_python/pull/977 to make
bazel coveragework with the hermetic Python toolchain from@rules_python. I found some issues whilst doing the work and I’ll copy them here for visibility. The issues were:coverage.pyv6.5.0 because the latest version (7.0.4has atypes.pyfile in the package directory, which imports from Python’s stdlib types [1]. Somehow the Python interpreter is thinking that thefrom types import FrameTypeis referring to the currently interpreted file and everything breaks. I would have expected the package to use absolute imports and only attempt toimport from coverage.typesif we usecoverage.typesand not just a plaintypesimport.multi_python_versionsexample cannot show coverage for the more complex tests that are usingsubprocessto spawn a different Python interpreter. I am wondering if this is related to the fact that we are includingcoverage.pyvia the toolchain and not through other mechanisms [2].__init__.pyfiles in the root of thebzlmodexample was breaking, when running under bazel coverage //:test. However, it started working when I renamed__init__.pytolib.py. I am suspecting that this has to do with the fact that the layer of indirection that coverage introduces could be something to do with that. Note thatbazel test ...works regardless of file naming.I think that all of these issues may be related to Python entrypoint template that is stored in this repository and not an issue with
rules_pythonitself.py_testfor pybind module does not work either. I expected it to work since the module is acc_binaryand hence gcov/lcov should be sufficient. All bazel commands -coverage|test|run <py-test-target>- produce empty coverage report.Interestingly running the
py-test-targetdirectly produces .gcda files as expected. So it seems bazel is simply ignoring the .gcda files for py_test targets? Has anyone seen this before and knows of a workaround?