php-code-coverage: ExecutableLinesFindingVisitor executable lines does not match the lines from xdebug coverage

Q A
php-code-coverage version 9.2.17 (latest)
PHP version 8.0.23 (latest)
Driver Xdebug
Xdebug version 3.2.0alpha3 (latest)
Installation Method Composer
Usage Method PHPUnit & other
PHPUnit version (if used) 9.5.24 (latest)

An issue, that took me quite a while to figure out what is going on.

In atk4/ui we have the following coverage setup:

a) collect coverage in phpunit tests, config: https://github.com/atk4/ui/blob/33779a9414b624a8faa7e959883d4ba407099dcd/phpunit.xml.dist#L23-L29 b) collect coverage from web requests, config: https://github.com/atk4/ui/blob/33779a9414b624a8faa7e959883d4ba407099dcd/demos/init-app.php#L21-L24 and https://github.com/atk4/ui/blob/33779a9414b624a8faa7e959883d4ba407099dcd/src/Behat/CoverageUtil.php#L38-L43 c) then merge all the coverage files using phpunit/phpcov in https://github.com/atk4/ui/blob/33779a9414b624a8faa7e959883d4ba407099dcd/.github/workflows/test-unit.yml#L223

Coverage collection is working. But as soon as we include tests/ dir also in b), the coverage decreases. This is wrong, as all tests/ files are already included in a) and inclusion of an already included file in another coverage collection cannot decrease the merged coverage.

CI repro: https://github.com/atk4/ui/pull/1852/commits - see 2nd and 3rd commit

https://app.codecov.io/gh/atk4/ui/blob/b762d62434df393aad170a7378d3d45876f93f5a/tests/JsTest.php (2nd commit coverage) https://app.codecov.io/gh/atk4/ui/blob/99f347b7d04567faefafa3b39105042b728e1649/tests/JsTest.php (3rd commit coverage)

It seems the problem is in https://github.com/sebastianbergmann/php-code-coverage/pull/892 or in https://github.com/sebastianbergmann/phpcov.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 37 (37 by maintainers)

Most upvoted comments

My best ipothesis is that the well known hack of PHPUnit to use include/require to parse unknown classes is messing things up here.

Please try these two changes together:

  1. Add processUncoveredFiles to your phpunit.xml
  2. Add self::$coverage->processUncoveredFiles(); to your Atk4\Ui\Behat\CoverageUtil::start()

And let’s see how the CI behaves.

The code provided produces two identical files, both with <line num="32" type="stmt" count="0"/>