msbuild: Exec tasks warns: bash : warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Steps to reproduce
Create an MSBuild project, that calls the Exec task to any bash script and run it in an environment where LC_ALL is not configured i.e a fresh ubuntu:16.04 docker container.
docker run -ti --rm ubuntu:16.04
apt-get update
apt-get install libicu-dev
dotnet SDK will need to be installed.
Project file
<Project>
<Target Name="RunExec">
<Exec Command="./helloworld.sh" />
</Target>
</Project>
helloworld.sh contents:
#!/usr/bin/env bash
echo 'Hello World'
Directory contents:
/
- project.csproj
- helloworld.sh
Command line
dotnet msbuild /t:RunExec
Expected behavior
Hello World
Actual behavior
bash : warning : setlocale: LC_ALL: cannot change locale (en_US.UTF-8) [/home/project.csproj]
Hello World
Environment data
msbuild /version output: 15.9.20.62856
dotnet SDK version: 2.2.103
OS info: Ubuntu 16.04 docker container.
If applicable, version of the tool that invokes MSBuild (Visual Studio, dotnet CLI, etc): dotnet CLI
I noticed that LC_ALL is set when calling the Exec task in Unix:
https://github.com/Microsoft/msbuild/blob/f136edbfecec171d73cb4c7fe1c936f6b2bf60a5/src/Tasks/Exec.cs#L597
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 4
- Comments: 43 (31 by maintainers)
Commits related to this issue
- Add locales to ubuntu 16.0.4 The MSBuild `Exec` task will use `export LC_ALL=en_US.UTF-8` as a part of the `bash` invocation. When ubuntu lacks the `locales` package this will cause a warning to be e... — committed to dotnet/dotnet-buildtools-prereqs-docker by jaredpar 4 years ago
- fix: add missing local - dotnet/msbuild#4194 — committed to visualon/docker-builder by viceice 2 years ago
- Add locales to ubuntu 16.0.4 The MSBuild `Exec` task will use `export LC_ALL=en_US.UTF-8` as a part of the `bash` invocation. When ubuntu lacks the `locales` package this will cause a warning to be e... — committed to fortinjose916/buildtools-prereqs-dotnet by fortinjose916 4 years ago
- Fixed weird MSBUILD build error. See https://github.com/Microsoft/msbuild/issues/4194 — committed to opentap/SSH by AsgerIversen 4 years ago
- Add check to detect Exec ConsoleOutput pollution See dotnet/msbuild#4194 — committed to notpeelz/VersionInfoGenerator by notpeelz a year ago
- Exec: use C instead of en_US.UTF-8 to set the Exec locale. (#9391) The en_US locale can't be used on systems where it is not installed. This is common in container images. On such systems, setting... — committed to dotnet/msbuild by tmds 8 months ago
- Exec: stop setting a locale on Unix. (#9449) The en_US locale can't be used on systems where it is not installed. This is common in container images. On such systems, setting the locale to en_US.U... — committed to dotnet/msbuild by tmds 7 months ago
Any idea why msbuild forces the locale? It seems weird, I would expect it to just keep the existing locale set by the system.
I recently got an error from a docker trying to build a package, because PackageVersion was detected like this
which is invalid. So this is not just a warning.
I think we could potentially have avoided this by specifying
C.UTF-8instead ofen_US.UTF-8, becauseman 3 setlocalesaysI don’t have enough understanding to figure what that would change or break, though. Any UNIX loc experts who can say it’s a good or bad idea floating by?
I intend to work on this next week.
If possible, it would be best to leave LANG and LC_ALL unchanged, so that if the tool outputs any messages, then they can be in the developer’s preferred language.
The code that sets those environment variables came from https://github.com/dotnet/msbuild/pull/288 to fix https://github.com/dotnet/msbuild/issues/251. It’s not clear to me what originally went wrong with “paths with non-ASCII characters in them”. It looks like .NET Runtime on POSIX always encodes file names as UTF-8, based on StringMarshalling.Utf8 in Interop.Sys.Open. Is the intention that, if the tool outputs an error message that includes a file name, then the file name (which the tool presumably treats as a byte string without a specific encoding) and the rest of the message should all be in the same encoding, so that MSBuild can decode the entire output to a string? And because the file name is UTF-8, you want the rest to be UTF-8 as well. If that is the purpose, then perhaps MSBuild could first check whether the environment variables already specify UTF-8 according to the C runtime library, and change them only if they don’t.
macOS 14 Sonoma does not recognize
C.UTF-8; the output oflocale -adoes not include it, and runningLANG=C.UTF-8 perl -e ''shows an error about the locale. macOS Sonoma supportsen_US.UTF-8.This is a good option. It will get rid of the warnings and still set the encoding to UTF-8.
Confirmed that this can be fixed by doing the following: