runtime: MSBuild Exec task broken with .NET Core 2.1

I’m running into an issue updating the .NET CLI to use .NET Core 2.1. My best guess currently is that this is a regression in .NET Core, because it is surfacing when I update to .NET Core 2.1 but worked with .NET Core 2.0.

Repro steps

On a non-Windows OS
With a version of the .NET CLI that uses .NET Core 2.1
Run dotnet msbuild with the following project

<Project>
    <Target Name="Build">
        <Exec Command="echo Hello World" />
    </Target>
</Project>

Expected

Build succeeds

Actual

Build fails with the following:

Microsoft ® Build Engine version 15.3.409.57025 for .NET Core Copyright © Microsoft Corporation. All rights reserved.

/bin/sh: 1: export LANG=en_US.UTF-8; export LC_ALL=en_US.UTF-8; . /tmp/tmp365c3d61708c464b9358f5adcdafd024.exec.cmd: not found /mnt/c/git/dotnet-cli-linux/artifacts/testexec/testexec.proj(6,9): error MSB3073: The command “echo Hello World” exited with code 127.

Details

The MSBuild Exec task creates a script file in the temporary folder in order to set encodings for the launched process. After the command is executed, the temporary file is deleted.

It appears that with .NET Core 2.1, the temporary script file is not visible to the launched process, resulting in a “not found” error.

Building a CLI with .NET Core 2.1

Currently there isn’t a version of the CLI available with .NET Core 2.1. To build one yourself in order to repro the issue:

Clone https://github.com/dsplaisted/cli
Check out the update-runtime-to-2.1 branch
Set the CLIBUILD_SKIP_TESTS environment variable to true
Run build.sh from the repo root
To use the newly built CLI, run dotnet from the artifacts/linux-x64/stage2 folder

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 1
Comments: 56 (55 by maintainers)

Commits related to this issue

Disable tests on non-Windows due to bugs related to .NET Core 2.1 https://github.com/dotnet/corefx/issues/23496 https://github.com/dotnet/cli/issues/7501 — committed to dsplaisted/cli by dsplaisted 7 years ago
Pin Stage0 to a version with .NET Core 2.0 instead of 2.1 This will avoid https://github.com/dotnet/corefx/issues/23496, which https://github.com/dotnet/cli/pull/7506 will work around — committed to dsplaisted/cli by dsplaisted 7 years ago
Pin Stage0 to a version with .NET Core 2.0 instead of 2.1 This will avoid https://github.com/dotnet/corefx/issues/23496, which https://github.com/dotnet/cli/pull/7506 will work around — committed to dsplaisted/cli by dsplaisted 7 years ago
Don't triple quote commands passed to the shell on Unix-like OS's This is due to a behavior change in .NET Core 2.1. The behavior of triple quotes used in Process.Start changed. This was deemed an ... — committed to dsplaisted/msbuild by dsplaisted 6 years ago
Don't triple quote commands passed to the shell on Unix-like OS's This is due to a behavior change in .NET Core 2.1. The behavior of triple quotes used in Process.Start changed. This was deemed an ... — committed to dsplaisted/msbuild by dsplaisted 6 years ago

Most upvoted comments

From offline discussions with @wtgodbe and @jkotas - we are not trying to say that /bin/sh has a bug. Rather that whoever passes triple quotes on Unix (msbuild in this case) has to be aware that single-quoted string will be passed over as argument (applying Windows rules) and it is up to the caller (msbuild in this case) to make sure the callee (/bin/sh in this case) can handle it. For /bin/sh as a callee it is invalid syntax to pass -c string with string in quotes - it is interpreted as commands and quotes are invalid there (see Invocation section). For another app (not shell) it could be valid.

The “problematic” angle is that due to our bug we didn’t handle multiple quotes correctly as we should have per Windows spec. We fixed it during 2.1 development cycle in dotnet/corefx#21339, which uncoreved this bug in msbuild (it depended on our incorrect behavior of stripping ALL triple quotes, instead of leaving single quotes in place). I believe that the fix dotnet/corefx#21339 is a good one, even though it is technical breaking change. Closing this issue as By Design / Won’t Fix – current behavior is the right one (given our past decisions to use Windows command line parsing rules also on Linux).

karelz on Dec 15, 2017

@ianhays @Priya91 - I verified this is an escaping issue between MSBuild and dotnet/corefx#21339, like you said @ianhays.

If you look at https://github.com/Microsoft/msbuild/blob/a9f64ebd108702c3fc65339c66cb124217854524/src/Tasks/Exec.cs#L609-L612, you’ll see MSBuild is appending triple quotes around the option

                commandLine.AppendSwitch("-c");
                commandLine.AppendTextUnquoted(" \"\"\"");
                commandLine.AppendTextUnquoted("export LANG=en_US.UTF-8; export LC_ALL=en_US.UTF-8; . ");
                commandLine.AppendFileNameIfNotNull(batchFileForCommandLine);
                commandLine.AppendTextUnquoted("\"\"\"");

If I remove one of those quotes, the issue no longer repros.

@rainersigwald @jeffkl - in case they know exactly why the Exec task is using triple quotes.

Either way, I think we’ve narrowed down the bug to the exact problem - using triple quotes passing into Process.StartInfo.Arguments.

eerhardt on Aug 24, 2017

This appears to be a problem with Linux shell, not with the way we parse args. To illustrate, consider the following program:

#include <unistd.h>

int main(int argc, char** argv[]) {
    char * const environment[] = {NULL};
    char * const args[] = { "-c", "\"echo.sh\"};
    execvpe("/bin/sh", args, environment);
}

This will fail with:

Can’t open “echo.sh”

This is analogous to the scenario in the repro, where we parse the triple-quoted stuff into an arg surrounded by single-quotes, then pass that directly to /bin/sh. If you run the same thing from the command line, the command line shell will parse the " out of the argument & pass it to /bin/sh in a format it can read (that is, without any quote characters). When /bin/sh receives a string surrounded by quotes, it treats them literally, and looks for an executable named "echo.sh" instead of echo.sh. As long as our argument parsing behavior is consistent with windows (which it is), then I’d consider this an issue with /bin/sh, and not with our Process APIs. Therefore I’m closing this as a non-issue.

wtgodbe on Dec 15, 2017

cc: @Priya91 in case she’s aware of any changes in that timespan that could cause the error e.g. maybe https://github.com/dotnet/corefx/pull/21339

ianhays on Aug 23, 2017