vstest: dotnet test does not forward MSBuild properties to msbuild in .NET 7 RC1
Description
Starting with .NET SDK 7.0.100-rc.1.22431.12
MSBuild parameters specified on the command line to dotnet test
are no longer forwarded to MSBuild.
The issue was not present in .NET 7 preview 7.
Steps to reproduce
Clone the martincostello/api repository at commit 61ec79c1c1e773b721f6ebdf28ed189d78d63ae7
and then run dotnet test
with the arguments specified below.
git clone https://github.com/martincostello/api
cd api
git checkout 61ec79c1c1e773b721f6ebdf28ed189d78d63ae7
dotnet test ./tests/API.Tests --configuration Release --filter Category=EndToEnd /p:CollectCoverage=false
Expected behavior
dotnet test
exits with an exit code of 0
.
Actual behavior
dotnet test
returns a non-zero exit code due to the CollectCoverage
property not being set to false
, causing the minimum coverage requirement of coverlet to not be satisfied.
+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+
| API | 0% | 0% | 0% |
+--------+------+--------+--------+
+---------+------+--------+--------+
| | Line | Branch | Method |
+---------+------+--------+--------+
| Total | 0% | 0% | 0% |
+---------+------+--------+--------+
| Average | 0% | 0% | 0% |
+---------+------+--------+--------+
C:\Users\martin.costello\.nuget\packages\coverlet.msbuild\3.1.2\build\coverlet.msbuild.targets(71,5): error : The minimum line coverage is below the specified 55 [C:\Coding\martincostello\api\tests\API.Tests\API.Tests.csproj] C:\Users\martin.costello\.nuget\packages\coverlet.msbuild\3.1.2\build\coverlet.msbuild.targets(71,5): error : The minimum branch coverage is below the specified 55 [C:\Coding\martincostello\api\tests\API.Tests\API.Tests.csproj]
C:\Users\martin.costello\.nuget\packages\coverlet.msbuild\3.1.2\build\coverlet.msbuild.targets(71,5): error : The minimum method coverage is below the specified 55 [C:\Coding\martincostello\api\tests\API.Tests\API.Tests.csproj]
C:\Users\martin.costello\.nuget\packages\coverlet.msbuild\3.1.2\build\coverlet.msbuild.targets(71,5): error : [C:\Coding\martincostello\api\tests\API.Tests\API.Tests.csproj]
C:\Users\martin.costello\.nuget\packages\coverlet.msbuild\3.1.2\build\coverlet.msbuild.targets(71,5): error : at Coverlet.MSbuild.Tasks.CoverageResultTask.Execute() in /_/src/coverlet.msbuild.tasks/CoverageResultTask.cs:line 247 [C:\Coding\martincostello\api\tests\API.Tests\API.Tests.csproj]
Environment
❯ dotnet --info
.NET SDK:
Version: 7.0.100-rc.1.22431.12
Commit: f1cf61e1c0
Runtime Environment:
OS Name: Windows
OS Version: 10.0.22000
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\7.0.100-rc.1.22431.12\
Host:
Version: 7.0.0-rc.1.22426.10
Architecture: x64
Commit: 06aceb7015
.NET SDKs installed:
2.1.818 [C:\Program Files\dotnet\sdk]
3.1.120 [C:\Program Files\dotnet\sdk]
3.1.201 [C:\Program Files\dotnet\sdk]
3.1.302 [C:\Program Files\dotnet\sdk]
3.1.303 [C:\Program Files\dotnet\sdk]
3.1.403 [C:\Program Files\dotnet\sdk]
3.1.423 [C:\Program Files\dotnet\sdk]
5.0.103 [C:\Program Files\dotnet\sdk]
5.0.104 [C:\Program Files\dotnet\sdk]
5.0.203 [C:\Program Files\dotnet\sdk]
5.0.214 [C:\Program Files\dotnet\sdk]
5.0.303 [C:\Program Files\dotnet\sdk]
5.0.408 [C:\Program Files\dotnet\sdk]
5.0.411 [C:\Program Files\dotnet\sdk]
6.0.100-rc.1.21458.32 [C:\Program Files\dotnet\sdk]
6.0.102 [C:\Program Files\dotnet\sdk]
6.0.109 [C:\Program Files\dotnet\sdk]
6.0.202 [C:\Program Files\dotnet\sdk]
6.0.203 [C:\Program Files\dotnet\sdk]
6.0.304 [C:\Program Files\dotnet\sdk]
6.0.400 [C:\Program Files\dotnet\sdk]
6.0.401 [C:\Program Files\dotnet\sdk]
7.0.100-rc.1.22431.12 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.0-preview.8.20414.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0-preview.7.22376.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.0-rc.1.22427.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.0-preview.8.20407.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.0-preview.5.21301.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.0-rc.1.21451.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.0-rc.2.21480.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0-preview.7.22375.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.0-rc.1.22426.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.28 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.0-preview.8.20411.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.0-preview.5.21301.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.0-rc.1.21451.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.0-rc.2.21501.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.0-preview.7.22377.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.0-rc.1.22427.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Other architectures found:
arm64 [C:\Program Files\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\arm64\InstallLocation]
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]
Environment variables:
Not set
global.json file:
C:\Coding\martincostello\api\global.json
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 17
- Comments: 34 (12 by maintainers)
Commits related to this issue
- Temporarily skip the test report while https://github.com/microsoft/vstest/issues/4014 — committed to corvus-dotnet/Corvus.JsonSchema by mwadams 2 years ago
- Workaround until .NET 7 RC2 is released: use environment variable to pass to MSBuild (https://github.com/microsoft/vstest/issues/4014) — committed to gimlichael/Cuemon by gimlichael 2 years ago
- Microsoft did not fix the bug till release date (https://github.com/microsoft/vstest/issues/4014) — committed to codebeltnet/classlib-savvyio by gimlichael 2 years ago
- Using env variables to msbuild, as of https://github.com/microsoft/vstest/issues/4014 — committed to andmos/BikeshareClient by andmos 2 years ago
- Bumping build image to dotnet 7 (#127) * Bump Microsoft.Extensions.DependencyInjection.Abstractions Bumps [Microsoft.Extensions.DependencyInjection.Abstractions](https://github.com/dotnet/runtime)... — committed to andmos/BikeshareClient by andmos 2 years ago
- Updating to dotnet 7 Using env variables to get codecoverage because of https://github.com/microsoft/vstest/issues/4014#issuecomment-1249681754 — committed to andmos/BikeDashboard by andmos 2 years ago
- Updating to dotnet 7 (#155) Using env variables to get codecoverage because of https://github.com/microsoft/vstest/issues/4014#issuecomment-1249681754 — committed to andmos/BikeDashboard by andmos 2 years ago
- Feature/anm csproj cleanup (#156) * Updating to dotnet 7 Using env variables to get codecoverage because of https://github.com/microsoft/vstest/issues/4014#issuecomment-1249681754 * Cleaning up... — committed to andmos/BikeDashboard by andmos 2 years ago
- Working on build, moved cli coverlet params to env variables (https://github.com/microsoft/vstest/issues/4014). — committed to HatTrickLabs/dbExpression by gwgrubbs 2 years ago
- :chart_with_upwards_trend: Temporary fix for build properties not getting passed through See coverlet-coverage/coverlet#1391 & microsoft/vstest#4014 — committed to JSkimming/Castle.Core.AsyncInterceptor by JSkimming 2 years ago
- Merge pull request #27961 from baronfel/handle-test-parser-forwarded-args Closes https://github.com/microsoft/vstest/issues/4014 — committed to dotnet/sdk by Evangelink 2 years ago
- :chart_with_upwards_trend: Temporary fix for build properties not getting passed through See coverlet-coverage/coverlet#1391 & microsoft/vstest#4014 — committed to JSkimming/Castle.Core.AsyncInterceptor by JSkimming 2 years ago
- All changes from the base net7 branch that don't involve working around microsoft/vstest#4014. — committed to SteveGilham/altcover by SteveGilham 2 years ago
- workaround dotnet test properties forwarding (microsoft/vstest#4014) — committed to Doraku/DefaultEcs by Doraku 2 years ago
- Fix tests coverage regression bug in .NET 7.0.100 The coverlet msbuild doesn't work and thus the tests coverage report is not generated. It is a temporarily bug cause in .NET 7.0.100 version and will... — committed to MichaelZaslavsky/social-event-manager by MichaelZaslavsky 2 years ago
- Fix tests coverage regression bug in .NET 7.0.100 The coverlet msbuild doesn't work and thus the tests coverage report is not generated. It is a temporarily bug cause in .NET 7.0.100 version and will... — committed to MichaelZaslavsky/social-event-manager by MichaelZaslavsky 2 years ago
- Work around issue in dotnet test See: https://github.com/microsoft/vstest/issues/4014 — committed to SonarSource/sonar-scanner-msbuild by costin-zaharia-sonarsource 2 years ago
- Work around issue in dotnet test See: https://github.com/microsoft/vstest/issues/4014 — committed to SonarSource/sonar-scanner-msbuild by costin-zaharia-sonarsource 2 years ago
- Work around issue in dotnet test See: https://github.com/microsoft/vstest/issues/4014 — committed to SonarSource/sonar-scanner-msbuild by costin-zaharia-sonarsource 2 years ago
- Work around issue in dotnet test See: https://github.com/microsoft/vstest/issues/4014 — committed to SonarSource/sonar-scanner-msbuild by costin-zaharia-sonarsource 2 years ago
@baronfel I’d like to leave the feedback that this is one of those large issues where the fix shouldn’t be delayed by over a month to coincide with fixed release schedules. This should have been fixed as fast as possible. Especially seeing all the comments here about people getting this issue pushed on them via updates of their github actions etc.
Hello folks! A quick update to let you know that the fix was just merged and will be available in
7.0.101
. Once again really sorry in the delay in fixing the issue!.NET SDK 7.0.101 was released today and I can confirm it fixed problems on our build server.
Thanks for getting the fix in. What surprises me is that a regression like this landed in the release in the first place. Especially since coverlet appears to be the de facto standard coverage provider and their official tutorial makes heavy use of this mechanic this should have been a release blocker, is there any reason why it wasn’t?
7.0.101 will be the first monthly servicing release of the 7.0.1xx SDK feature band, so it will arrive early/mid next month around the same time as all of our monthly servicing releases.
Hey there! Sorry for the long delay, we have an open PR with the fix that is waiting for validation and we will bring the subject to the table so we can push the release as fast as possible. Will ping back here as soon as it’s available.
As a workaround, since MSBuild accepts environment variables as property values you could set an env var for each
/p
property you needed for the duration of the command.Good spot, thanks for the ping @slang25. We’ll take a look and get this fixed for rc2.
FYI It seems to accepts the first msbuild property but nothing after. This is causing issues throughout all our pipelines! While the environment variable solution would work, we have to update all our shared and hardened AZDO templates (likely to revert once this is fixed) which we aren’t crazy about.
This should’ve been corrected for GA…
Yeah, I know it sucks, but a workaround (as mentioned by @baronfel) is to add the parameters as environment variables in your build pipeline; here is an Azure example from my own project:
If you want more context, have a look here: https://github.com/gimlichael/Cuemon/blob/development/.azure/steps/unit-test.yml
Is there an update on whether this issue will be fixed into time for .NET 7 GA?
Nope, the PR has not been merged 😢
Just to add to your voice: we see this issue too when kicking off tests from this PS task.
Thanks for the update @matyasbach. I will move forward by closing this issue as the fix is now released.
@heaths The work around is to not use .net 7.0.100, because that is where the bug originates.
This also breaks SqlClient tests. Is there an ETA on the fix?
Another option is to add the necessary properties to
csproj
/Directory.Build.props
, e. g.:Hi @JRahnama, the fix was done, we are just waiting for the releasing. For the ETA please see https://github.com/microsoft/vstest/issues/4014#issuecomment-1311751668
Hi, for the people could not wait for release 7.0.101. I was able to build the fixed version of dotnet.dll. You can download from: Here Just copy it in C:\Program Files\dotnet\sdk\7.0.100 and now it will work.
(Looks like there were changes to the way MSBuild parameters were passed through…)
https://github.com/dotnet/sdk/blob/release/7.0.1xx-preview7/src/Cli/dotnet/commands/dotnet-test/Program.cs
https://github.com/dotnet/sdk/blob/release/7.0.1xx-rc1/src/Cli/dotnet/commands/dotnet-test/Program.cs