php-code-coverage: Identification of executable lines for `match` expressions does not work correctly
| Q | A |
|---|---|
| php-code-coverage version | 9.2.21 |
| PHP version | 8.1.7 |
| Driver | Xdebug |
| Xdebug version (if used) | 3.1.5 |
| Installation Method | Composer |
| Usage Method | PHPUnit |
| PHPUnit version (if used) | 9.5.26 |
Some valuable precision in the identification of executable lines is lost as a result of #964. I have not reviewed that PR carefully enough to judge its merits, but I see how the concerns raised by @mvorisek can manifest as a regression in real-world test suites. I appreciate the many, many hours of effort by the contributors working on this issue and I hope that we can find a solution that is still friendly to mutation testing without impacting precision.
For example, in 9.2.20, only the branches of a match expression that were actually executed were displayed as covered. In 9.2.21, the entire match expression is now reported as being covered, even though it is not. It was quite useful to know which branches were being exercised by the test.
I’m having a bit of deja vu here, because I reported a nearly identical regression to match expression coverage in #904, which was fixed somewhere between 9.2.13 and 9.2.20! 😃
Here is an example class:
<?php declare(strict_types=1);
class MatchExpr {
public int $result;
public function __construct(int $value) {
$this->result = match ($value) {
0 => 4,
1 => 5,
2 => 6,
3 => 7,
default => 8,
};
}
}
And here is the corresponding test:
<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;
use MatchExpr;
/**
* @covers MatchExpr
*/
class MatchExprTest extends TestCase {
/**
* @testWith [0, 4]
* [2, 6]
* [9, 8]
*/
public function test_happy_path(int $value, int $expected): void {
self::assertSame($expected, (new MatchExpr($value))->result);
}
}
Coverage report in 9.2.20 (only 3 of the 5 branches are covered, as expected):

Coverage report in 9.2.21 (all branches are marked as covered):

Thank you for your time!
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 20 (18 by maintainers)
Commits related to this issue
- Add test for #967 — committed to sebastianbergmann/php-code-coverage by sebastianbergmann 2 years ago
- Closes #967 — committed to sebastianbergmann/php-code-coverage by sebastianbergmann 2 years ago
Given the following command:
For which I need to highlight 2 things:
(new Foo)->foo("HIT");And the following directory structure, the
Fooclass receive from xDebug the information in the comment at the end of each line:So despite only asking for
'HIT'match, all other cases are marked as executed too when they precede the one we are looking for, and also thedefaultone is triggered even if it’s at the end.Line Code-Coverage for
matchis a total lie.Activating Infection requires 3 steps:
composer require --dev infection/infectionvendor/bin/infection --git-diff-lines --min-msi=100to only run it on DIFFIf you set up PHPUnit and Code-Coverage, you certainly have the means for it. If you care about
matchCC precision, you certainly need it.I understand the psychological burden of adding another tool to the build. I can only reply that I didn’t choose to have such poor Code-Coverage tools from
php-srcin the first place neither.@hemberger as far as I can tell, this only affect
matchby the way, so I’d appreciate if you can change the title to indicate a specificmatchregression, so other people that face the same issue can find this reply quicker.Finally found some time to work on this today, sorry for the delay.
Based on the example in https://github.com/sebastianbergmann/php-code-coverage/issues/967#issue-1497664268, I have created the following reproducing example:
src/MatchExpr.phptests/MatchExprTest.phpphpunit.xmlStatus Quo
Using php-code-coverage at revision a713584642ec65ccc5327325a485b5c6d8a1cd4e, I get this:
Patch
When I revert the code changes from ede00c743467b452a005bcad457887c5012bf4ec, I get this:
What strikes me as odd is the fact that no test for php-code-coverage fails when I just apply the patch shown above. But I will not go down the rabbit hole to find out why that is. Instead, I will add a new test case based on the example shown above.
As far as I remember you are correct
There seems to be a bug in VLD, it doesn’t show the “default” jump target in the list of targets (from the original report):
The coverage from PHP-CC 9.2.20 looks correct, as cases 0, 2, and the default are tested, and shown in line coverage. I don’t know what you have done in 9.2.21, where it is broken.
That seems to be because PHP checks whether the return type of the
returnstatement of the whole function on that line. It’s not wrong, but confusing. It would be better if that was done on the actualreturn matchline — but this is something for PHP to change.That’s a nonsense statement.
In Slamdunks example, there is no jump table, as the LHS has code running required for evaluations. This is similar to how switch works when it can’t use a jump table either:
As this code is run, case by case until it matches, these LHS statements are evaluated/executed, so it shows up correctly in line coverage.
The example doesn’t talk about, or show, branch coverage, so I can’t comment on that.
So the only bug here, is that VLD misses showing the default case, and I’ve just fixed that.