vstest: dotnet test does not respect LogFileName -- adds date

From @hpbieker on March 11, 2019 9:59

Steps to reproduce

  • Install .NET Core 3.0 preview3
  • Create a test project
  • Run dotnet test --logger 'trx;LogFileName=somename.trx'

Expected behavior

A file named somename.trx should be created. In .NET Core 2.2 / Test Execution Command Line Tool Version 15.9.0 I get this behaviour.

Actual behavior

A file named somename_2019-03-11_10-38-36-764.trx was created.

$ dotnet test --logger 'trx;LogFileName=somename.trx'
Test run for C:\Users\nohabie\AppData\Local\Temp\test\bin\Debug\netcoreapp3.0\test.dll(.NETCoreApp,Version=v3.0)
Microsoft (R) Test Execution Command Line Tool Version 16.0.0-preview-20190124-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Results File: C:\Users\XXX\AppData\Local\Temp\test\TestResults\somename_2019-03-11_10-38-36-764.trx

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1,1557 Seconds

Environment data

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview3-010431
 Commit:    d72abce213

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 RID:         win7-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.0.100-preview3-010431\

Host (useful for support):
  Version: 3.0.0-preview3-27503-5
  Commit:  3844df9537

.NET Core SDKs installed:
  2.1.600 [C:\Program Files\dotnet\sdk]
  2.2.200 [C:\Program Files\dotnet\sdk]
  3.0.100-preview3-010431 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview3-19153-02 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview3-27503-5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0-preview3-27504-2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Copied from original issue: dotnet/cli#10962

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 13
  • Comments: 25 (7 by maintainers)

Commits related to this issue

Most upvoted comments

@hpbieker this was done when dotnet test support at solution level was added i believe. The scenario was that when you ran it at a solution level multiple csprojs are present and multiple trx files are generated. To ensure these don’t override each other this change was made.

I’m sorry this broke your build scripts. But I don’t see a way to go back to honoring the exact file name without adding a post fix since it will break the multi csproj scenario.

You can look for trx files with a regex that checks for the name you have as the prefix. (This is what we do in the AzureDevops pipelines, to publish the trx-es). I hope the explanation was satisfactory?

+31
ShreyasRmsft on Mar 14, 2019

But it did not behave this way in In .NET Core 2.2 / Test Execution Command Line Tool Version 15.9.0 I get this behaviour? This change broke my build scripts because the trx file was not found anymore.

+23
hpbieker on Mar 12, 2019

BTW: this happens even when running tests against a single .csproj file. We have the same problem where it broke our CI pipes.

+12
brittonbeckham on Apr 3, 2019

This also broke all of our build pipelines too. Nothing exotic is happening in this section of our pipelines - it’s merely pulling the microsoft/dotnet:2.2-sdk Docker image and running dotnet vstest /logger:"trx;LogFileName=report.unit.xml", so it looks like this was a breaking change made in a non-major version update…?

+9
jonjomckay on Apr 4, 2019

+1, broke our build chain too. I’m surprised such a change was implemented without any backwards compatibility. Was it not more sensible to implement a “UniqueLogFileName” parameter for those people with the multi-target issue to use, and to leave the behaviour of “LogFileName” untouched/unbroken?

+7
andyturner on Apr 15, 2019

@ShreyasRmsft This change broke our pipelines too.

Is it normal that a patch version (.NET Core SDK 2.1.602) brings a breaking change like that?

Workaround: change your build script or use SDK version 2.1.505.

Note that we used the microsoft/dotnet:2.1-sdk docker image available on Docker Hub. Since that image point to 2.1.602 instead of 2.1.505 it broke our pipelines.

+7
tremblaysimon on Apr 4, 2019

This broke our build scripts that execute inside of Jenkins. The test result collector at the end of our build pipeline stages can no longer find the files. We have many, many dozens of these build scripts, all run inside Docker images. Fixing this is a huge issue for us, and a reason not to adopt the newer .NET Core versions.

We also have many multi-test solutions in place. Our pattern is to run each test project separately and collect the result files at the end of the stage. Your “multi test” solution is for a problem that many developers have already worked around.

The better solution would have been to allow developers the options to change the suffix if we had needed to, rather than forcing it on us and breaking our builds without warning.

Your solution to a problem that didn’t exist is bad.

+6
crhairr on Apr 16, 2019

I think MS wants us to open another issue for this, but seems like it shouldn’t have been closed. A patch like this to a container image shouldn’t have been a breaking change without, at minimum, a minor version number change. We found a work around for the meantime, but this is definitely frustrating.

On the flip side, I am happy and excited about where all this .NET Core stuff is going and have been using it heavily. Having used .NET since v1, i can say I have never been more excited to code in .NET Core. Cheers to the entire team!

+5
brittonbeckham on Apr 8, 2019

@brittonbeckham @Marusyk @Kemyke Apologies for the break here and do acknowledge that we should have called this out broadly and with an equivalent version update. The cause of the change was to address the scenario of multi targeted test project where the log file provided was being overwritten and ending up with data loss. We will revisit the change and see if we could retain the earlier behavior and at the same time address the issue with multi target test projects.

+3
cltshivash on Apr 9, 2019

It was fixed via https://github.com/microsoft/vstest/pull/1996 Release: https://github.com/microsoft/vstest/releases/tag/v16.0.2-preview-20190502-01 Notes: https://github.com/Microsoft/vstest-docs/blob/master/docs/releases.md#1602-preview-20190502-01

+2
mayankbansal018 on May 13, 2019

@livarcocc as of today this is by design.

We do this to create unique files. Consider the option as more of a logFilePrefix rather than logFileName.

And changing the param name is contract breaking change so we won’t be able to rename it.

+2
ShreyasRmsft on Mar 12, 2019
Read more comments on GitHub
← vstest: dotnet test does not forward MSBuild properties to msbuild in .NET 7 RC1
vstest: `dotnet test` does not work for projects with custom output path →