tarpaulin: --follow-exec causes difference in execution

Describe the bug In some cases it appears using --follow-exec can cause test code to not run somehow.

To Reproduce https://github.com/boustrophedon/tarpaulin_missing_coverage/commits/master

The original problem I’m trying to solve is the following: I have tests in tests/ and examples in examples/ and I want to get combined coverage for all of them. If I just run cargo tarpaulin --tests --examples, the examples don’t execute because the test runner is used to look for tests rather than execute the examples (related?). I’ve worked around this by adding a small test in each example that just calls main.

However, one of my example programs calls itself (which, as above, is actually the test runner process) as a subprocess and although it appears the processes are executing (e.g. if I intentionally throw a panic in one, the panic shows up), tarpaulin isn’t catching that.

While investigating this I’ve found a minimal example that shows just by adding and removing the --follow-exec flag that the subprocess code isn’t being called somehow.

Expected behavior The subprocess code should run, the file should be written to, and the CI step “Check file was actually created” should pass.

fails: https://github.com/boustrophedon/tarpaulin_missing_coverage/commit/9d91fb202ae08b575528e6aa13c3404e954cd0c9 succeeds: https://github.com/boustrophedon/tarpaulin_missing_coverage/commit/4e92bfe238d5391eac58f3c562b8316796cb4655

About this issue

Original URL
State: open
Created 2 years ago
Comments: 28 (14 by maintainers)

Commits related to this issue

Add test for issue #966 — committed to xd009642/tarpaulin by xd009642 2 years ago

Most upvoted comments

I’ll do it when the PR’s merged, that way you’ll know the release is coming imminently and can switch your CI to use the latest release 👍

xd009642 on Mar 3, 2022

Brilliant, I’ll be tweaking the branch a bit more to try and fix the issue it was originally there to fix. But once it’s merged in I’ll cut a new release 👍

xd009642 on Mar 3, 2022

Okay, If you try now it works with and without the config file! Finally satisfied I’ve laid this one to rest. And I think this may have done enough to unblock me on #953 so thanks 😁

xd009642 on Mar 3, 2022

so for the first part, cargo tarpaulin --examples is equivalent to cargo test --examples which is because of the issue you linked. If you want to avoid creating example tests you can run the examples directly with cargo tarpaulin --command Build --examples and use a config file to combine that with other test types. Something as follows should work:

[examples]
command = "Build"
run-types = ["Examples"]

[tests]
run-types =["tests"] # Just put the others here - maybe an empty section will work but I haven't tried

I have a feeling this may be related to some things I’ve been seeing with https://github.com/xd009642/tarpaulin/issues/953 so will try the examples and see if i can puzzle it out

xd009642 on Feb 28, 2022