coverlet: Deterministic build, Coverlet throws a KeyNotFoundException when ContinuousIntegrationBuild = true.

Hello, when I try to run Coverlet (MSBuild or VSTest integration) with UseSourceLink = true and ContinuousIntegrationBuild = true, it throws a KeyNotFoundException.

Data collector 'XPlat code coverage' message: [coverlet]Coverlet.Collector.Utilities.CoverletDataCollectorException: CoverletCoverageDataCollector: Failed to get coverage result
 ---> System.Collections.Generic.KeyNotFoundException: The given key '' was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Coverlet.Core.Coverage.GetSourceLinkUrl(Dictionary`2 sourceLinkDocuments, String document) in /_/src/coverlet.core/Coverage.cs:line 452
   at Coverlet.Core.Coverage.CalculateCoverage() in /_/src/coverlet.core/Coverage.cs:line 350
   at Coverlet.Core.Coverage.GetCoverageResult() in /_/src/coverlet.core/Coverage.cs:line 141
   at Coverlet.Collector.DataCollection.CoverageWrapper.GetCoverageResult(Coverage coverage) in /_/src/coverlet.collector/DataCollection/CoverageWrapper.cs:line 44
   at Coverlet.Collector.DataCollection.CoverageManager.GetCoverageResult() in /_/src/coverlet.collector/DataCollection/CoverageManager.cs:line 93
   --- End of inner exception stack trace ---
   at Coverlet.Collector.DataCollection.CoverageManager.GetCoverageResult() in /_/src/coverlet.collector/DataCollection/CoverageManager.cs:line 98
   at Coverlet.Collector.DataCollection.CoverletCoverageCollector.OnSessionEnd(Object sender, SessionEndEventArgs e) in /_/src/coverlet.collector/DataCollection/CoverletCoverageCollector.cs:line 160.

Here is a very simple repository that reproduces the error. The exact commands that I use are:

dotnet test /p:CollectCoverage=true /p:UseSourceLink=true /p:ContinuousIntegrationBuild=true

and

dotnet test --collect:"XPlat Code Coverage" --settings coverlet.USL.runsettings /p:ContinuousIntegrationBuild=true

If I use ContinuousIntegrationBuild = false, everything works fine.

Context:

.NET Core v3.1.300+
coverlet.collector v1.3.0 or coverlet.msbuild v2.9.0 with patch for CoverletGetPathMap
Microsoft.SourceLink.GitHub v1.0.0

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 20 (1 by maintainers)

Most upvoted comments

Thanks @chtoucas can repro on my local, I’ll take a look asap, sorry guys but are very busy months I’m near prod release on my daily job so you know what I mean 🙏 BTW found the issue, fix shouldn’t be so hard, confirm that the problem is that deterministic build generates “deterministic” path inside sourcelinks metada, so we need to translate also those. Dev notes https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Coverage.cs#L356

MarcoRossignoli on Jul 10, 2020

There are two issues that 300 fixed:

the need for the workarounds in for the generated files
enabling the base targets to return the SourceRoot items so the coverlet target workarounds aren’t needed.

@MarcoRossignoli were you able to update coverlet to use the built-in target to get the SourceRoot items instead of requiring people to add something to their directory targets?

clairernovotny on Jul 13, 2020

We planned to release evey quarter, this time we release before for deterministic feature, we could release a fix or a new version as planned(I’ve a PR ready with a new feat also --skipautoprops). I need to talk to @tonerdo (we have some versions change planned) but not clear agreement yet . https://github.com/coverlet-coverage/coverlet/issues/874 https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/ReleasePlan.md#release-calendar

I’ll let you know, for now you can use nightly.

MarcoRossignoli on Jul 13, 2020

@MarcoRossignoli just finished the tests I had set up, and everything seems to work as it should with the latest nightly.

Tried both with and without the workaround mentioned here. Without the workaround, no error would be shown, but the generated report would be empty (I expected that to happen). With the workaround, everything would work as expected, with no error, correct coverage report, and so on.

NOTE: These tests were run using the coverlet MSBuild version

AdmiringWorm on Jul 12, 2020

@MarcoRossignoli we (or rather @AdmiringWorm) have a test ready to go. We will execute this once the latest nightly version is available. Thanks for looking into this, really appreciate it!

gep13 on Jul 11, 2020

Hi guys can someone of you give it a try using our nightly(tomorrow we release every midnight)? https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/ConsumeNightlyBuild.md

MarcoRossignoli on Jul 11, 2020

Thanks for the test!

MarcoRossignoli on Jul 6, 2020

@MarcoRossignoli as requested on Twitter here:

https://twitter.com/MarcoRossignoli/status/1280043730960093184

I have updated my build to no longer pass in /p:UseSourceLink=true and I can confirm that the build that was failing, is now succeeding, and coverage reports are now successfully being pushed to codecov.io, where they were failing before.

Thanks for the tip. Let me know if there is anything that I can do to help with the testing of this issue.

gep13 on Jul 6, 2020