moby: ANSI Code Page in Hyper-V Windows containers should not be 65001
Description When running a Windows container with Hyper-V isolation, the system ANSI Code Page (ACP) is set to 65001, which corresponds to the UTF-8 code page, rather than to an “ordinary” code page.
For Windows Desktop, this is a Beta feature called “Beta: Use Unicode UTF-8 for worldwide language support”:
As this causes incompatibilities with software that doesn’t support UTF-8, it should not be enabled in containers by default.
- Most notoriously, this happens with SQL Server: https://github.com/microsoft/mssql-docker/issues/466#issuecomment-506079975
- But it also affects other software: https://social.msdn.microsoft.com/Forums/en-US/0c4a2452-efd5-4e39-bd58-9748201e1670/codepages-in-servercorenanoserver?forum=windowscontainers
- Another example: https://github.com/conda/conda/issues/8088
Repro steps without SQL Server:
- Execute
docker run -it --rm --isolation=hyperv mcr.microsoft.com/windows/servercore:1903 powershell
- Inside the PowerShell, run this:
$Signature = "[DllImport(`"kernel32.dll`")] public static extern int GetACP();";$acpGetter = Add-Type -MemberDefinition $Signature -Name "Acp" -Namespace Win32Functions -PassThru; $acpGetter::GetACP()
- Observe the results
Describe the results you received:
- With process isolation: 1252 (an “ordinary” code page)
- With Hyper-V isolation: 65001 (the UTF-8 code page)
Describe the results you expected: The selected code page shouldn’t be determined by a beta feature that’s not enabled on the host, regardless of the isolation mode. So it should either
- be the same as on the host OS (in my case, 1252),
- be hardcoded to something more friendly to non-UTF-8 applications,
- or be configurable at container start.
Output of docker version
:
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:22:37 2019
OS/Arch: windows/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.24)
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:36:50 2019
OS/Arch: windows/amd64
Experimental: false
Output of docker info
:
Client:
Debug Mode: false
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 7
Server Version: 19.03.5
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics internal l2bridge l2tunnel nat null overlay private transparent
Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 18362 (18362.1.amd64fre.19h1_release.190318-1202)
Operating System: Windows 10 Pro Version 1903 (OS Build 18362.592)
OSType: windows
Architecture: x86_64
CPUs: 12
Total Memory: 31.85GiB
Name: W1119
ID: 6SEL:LRZQ:3NZL:TEA3:KE36:A6UU:Y4WU:EXDA:PRKO:QSZQ:QQA3:V4WU
Docker Root Dir: D:\DockerData
Debug Mode: true
File Descriptors: -1
Goroutines: 28
System Time: 2020-02-17T15:21:58.5160558+01:00
EventsListeners: 1
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 7
- Comments: 19 (5 by maintainers)
We are aware of this behavior - there isn’t any simple fix/workaround unfortunately, but we are looking at options.
Just for edification - the typical Windows installation (Windows Server 2019/2022, Windows 10/11) boots with the kernel configured for ASCII code page. However, the Hyper-V utility VM leverages a new composition method (partly what allows us to boot such a small image) boots with the kernel configured for UTF. It’s a none-insignificant change so we’re having to look at how to address the app compat delta while preserving size/agility.
Related: https://gitlab.kitware.com/cmake/cmake/issues/19102
CMake’s libarchive dependency received a patch to support the UTF-8 locale and fix a possible buffer overflow. If you ran into a CMake error involving
cmake -E tar
because of the Hyper-V utility VM code page being locked to UTF-8, then try CMake v3.17.0-rc2 or the upcoming v3.16.5 which contain the libarchive fix.Nonetheless, I’m sure there is a bunch of other software which has trouble with cp 65001 at Docker image build time and a fix / added ability to change the code page would be much appreciated.
I agree that it should be possible to isolate these settings from the host operating system. Unfortunately
RUN powershell -Command Set-WinSystemLocale de-DE
in the dockerfile has no effect at all, this is similar to the above complains, that setting the codepage, e.g. withchcp 1252
is impossible and it remains on 65001 (utf-8).With
# docker run --isolation=hyperv ...
there is actually no possibility to change these localization settings, which makes it unusable for specific applications, where localization has an impact.