sdk: Unable to test non-Core code on Linux

(Apologies if this is the wrong repo for this; it’s not clear to me whether this is xUnit-specific or the expected behaviour when testing on Linux.)

Background

I have projects which don’t target core yet, but I still want to test on Travis. They work fine on RC1 (with dnx test and the old xUnit setup) and work fine with an updated project.json on Windows, but fail to run tests on Linux.

It’s not clear whether the non-core runtimes/frameworks are expected to work on non-Windows platforms. Using dotnet run works fine for a console app with a TFM of net451, so it’s clearly got some knowledge… but dotnet test fails.

Similar issues: dotnet/sdk#4833, dotnet/sdk#5172.

Steps to reproduce

Create xUnit test project with a framework of net451, and run dotnet restore, dotnet build, dotnet test.

Sample project.json:

{
  "dependencies": {
    "xunit": "2.1.0",
    "dotnet-test-xunit": "1.0.0-rc2-build10015",
  },

  "testRunner": "xunit",

  "frameworks": {
    "net451": { }
  }
}

Sample C#:

using Xunit;

public class Test
{
    [Fact] public void Foo() {}
}

Expected behavior

Tests run (using Mono).

Actual behavior

Restore and build both work, but test fails:

dotnet-test Error: 0 : Microsoft.DotNet.Cli.Utils.CommandUnknownException: No executable found matching command "dotnet-test-xunit"  
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.FindProjectDependencyCommands(String commandName, IEnumerable`1 commandArgs, String configuration, NuGetFramework framework, String outputPath, String buildBasePath, String projectDirectory)  
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.Create(String commandName, IEnumerable`1 args, NuGetFramework framework, String configuration)  
  at Microsoft.DotNet.Tools.Test.ConsoleTestRunner.DoRunTests(ProjectContext projectContext, DotnetTestParams dotnetTestParams)  
  at Microsoft.DotNet.Tools.Test.TestCommand.DoRun(String[] args)

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  15.10
 OS Platform: Linux
 RID:         ubuntu.15.10-x6

About this issue

  • Original URL
  • State: open
  • Created 8 years ago
  • Reactions: 1
  • Comments: 32 (6 by maintainers)

Commits related to this issue

Most upvoted comments

I has been some time since this issue was raised. Is the above nasty workaround the only real solution at this point? The alternative being to wait for a preview3 tooling release?

If a net451 exe exists in the package, you will have to do a mono bootstrap script.

#!/usr/bin/env bash

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
  SOURCE="$(readlink "$SOURCE")"
  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"

exec mono "$DIR/dotnet-test-xunit.exe" "$@"

@borgdylan Thanks for your script. I’m using, with the following improvements:

  • Including your script at my test projects root directory

  • Putting the following configuration on projector.json:

    "buildOptions": {
        "copyToOutput": "dotnet-test-xunit"
    }
    
  • Adding the following line to .gitattributes:

    dotnet-test-xunit -text
    

This way, your script always gets copied to output directory, and I’m able to run test as I would normally would: dotnet test

dotnet xunit is removed starting from xunit 2.4 Ref: Release Notes 2.4 Excerpt from the Release Notes:

Unfortunately, this release also removes the dotnet xunit runner, as the stability of the runner was never perfect, and it suffered from many assembly-loading related issues. Users from .NET Core can continue to use VSTest (either inside Visual Studio, or via dotnet test).

So, for xunit framework test use the command

dotnet test

See Getting Started with xUnit.net Multi-targeting with non-Windows OSes

@AdamPD if you can upgrade your xunit project up to 2.3.0-beta3 you can use the dotnet-xunit tool which I added support for mono https://www.nuget.org/packages/dotnet-xunit/ https://github.com/xunit/xunit/pull/1213

The bash script workaround no longer works when upgrading to the latest .NET SDK 1.0.1. The tests do run when using mono xunit.console.exe test.dll, but dotnet test fails to find any “again”, specifying “‘TestAdapterPath’ does not contain any test adapters”, continues with “permission denied”, then exits.

I confirm that dotnet test does NOT work on linux when targetting net451.

Setup:

  • Debian 8 (x64)
  • Mono 4.2.3
  • Dotnet 1.0.0-preview1-002702

project.json:

{
  "version": "1.0.0-*",

  "testRunner": "xunit",

  "dependencies": {
    "xunit": "2.1.0",
    "dotnet-test-xunit": "1.0.0-rc3-build10019"
  },

  "frameworks": {
    "net451": { }
  }
}

Output of dotnet test:

sebok$ dotnet test
Project Atelier.Tests (.NETFramework,Version=v4.5.1) was previously compiled. Skipping compilation.
dotnet-test Error: 0 : Microsoft.DotNet.Cli.Utils.CommandUnknownException: No executable found matching command "dotnet-test-xunit"
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.FindProjectDependencyCommands(String commandName, IEnumerable`1 commandArgs, String configuration, NuGetFramework framework, String outputPath, String buildBasePath, String projectDirectory)
  at Microsoft.DotNet.Cli.Utils.ProjectDependenciesCommandFactory.Create(String commandName, IEnumerable`1 args, NuGetFramework framework, String configuration)
  at Microsoft.DotNet.Tools.Test.ConsoleTestRunner.DoRunTests(ProjectContext projectContext, DotnetTestParams dotnetTestParams)
  at Microsoft.DotNet.Tools.Test.TestCommand.DoRun(String[] args)

However, the build created directory bin/Debug/net451/debian.8-x64, from which I can manually run dotnet-test-xunit.exe:

sebok$ cd bin/Debug/net451/debian.8-x64/
sebok$ mono dotnet-test-xunit.exe Atelier.Tests.dll
xUnit.net .NET CLI test runner (64-bit debian.8-x64)
  Discovering: Atelier.Tests
  Discovered:  Atelier.Tests
  Starting:    Atelier.Tests
  Finished:    Atelier.Tests
=== TEST EXECUTION SUMMARY ===
  Atelier.Tests  Total: 1, Errors: 0, Failed: 0, Skipped: 0, Time: 0,063s

Got it works with work around on Ubuntu 14.04 Project:

{
  "version": "1.0.0-*",

  "testRunner": "xunit",

  "dependencies": {
    "dotnet-test-xunit": "1.0.0-rc3-*",
    "Project": "*",
    "xunit": "2.1.0"
  },

  "frameworks": {
    "net451": {
      "dependencies": {
        "Microsoft.NETCore.Platforms": "1.0.1-rc2-*",
        "Moq": "4.2.1510.2205"
      }
    }
  }
}

Failed: dotnet test test/ProjectFolderTest

Success: mono test/ProjectFolderTest/bin/Debug/net451/ubuntu.14.04-x64/dotnet-test-xunit.exe test/ProjectFolderTest/bin/Debug/net451/ubuntu.14.04-x64/ProjectFolderTest.dll

You’re missing the testRunner entry. This may help: http://xunit.github.io/docs/getting-started-dotnet-core.html