WSL: gzip from Ubuntu Jammy doesn't execute
Version
Microsoft Windows [Version 10.0.19044.1586]
WSL Version
- WSL 2
- WSL 1
Kernel Version
4.4.0-19041-Microsoft
Distro Version
Ubuntu 22.04 “Jammy Jellyfish”
Other Software
GZip version 1.10-4ubuntu3 and 1.10-4ubuntu4 amd64.
Repro Steps
apt install gzip=1.10-4ubuntu3then rungzip.apt install gzip=1.10-4ubuntu4then rungzip.
Expected Behavior
No error shows up.
Actual Behavior
sh: 1: gzip: Exec format error
The binary doesn’t execute, so no strace.
Diagnostic Logs
- Report on Launchpad: LP#1966849
- Possibly related: #4461.
Similar to that one, the same binary runs perfectly OK on a native Ubuntu Jammy machine. However, this time the binary is 97520 bytes and no section points outside this range.
Dissected binary using Wireshark: https://paste.ubuntu.com/p/nc2v6ZSRHW/
The previous version gzip 1.10-4ubuntu1 is fine, so I’ve installed that one instead and setting apt-mark hold for now.
It’s very interesting that only gzip is found problematic. And it’s the same program as 3 years ago. Time to wonder if gzip has any magic to break on WSL1.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 30
- Comments: 33 (1 by maintainers)
Links to this issue
Commits related to this issue
- add https://gist.github.com/the-moog/179753aacc52b262d1068878d520e9c9 script to fix 'exec format error' due to https://github.com/microsoft/WSL/issues/8219#issuecomment-1133936081 — committed to ysoftwareab/yplatform by andreineculau 4 years ago
- add https://gist.github.com/the-moog/179753aacc52b262d1068878d520e9c9 script to fix 'exec format error' due to https://github.com/microsoft/WSL/issues/8219#issuecomment-1133936081 — committed to ysoftwareab/yplatform by andreineculau 4 years ago
- add https://gist.github.com/the-moog/179753aacc52b262d1068878d520e9c9 script to fix 'exec format error' due to https://github.com/microsoft/WSL/issues/8219#issuecomment-1133936081 — committed to ysoftwareab/yplatform by andreineculau 4 years ago
- add https://gist.github.com/the-moog/179753aacc52b262d1068878d520e9c9 script to fix 'exec format error' due to https://github.com/microsoft/WSL/issues/8219#issuecomment-1133936081 — committed to ysoftwareab/yplatform by andreineculau 4 years ago
- add https://gist.github.com/the-moog/179753aacc52b262d1068878d520e9c9 script to fix 'exec format error' due to https://github.com/microsoft/WSL/issues/8219#issuecomment-1133936081 — committed to ysoftwareab/yplatform by andreineculau 4 years ago
- add https://gist.github.com/the-moog/179753aacc52b262d1068878d520e9c9 script to fix 'exec format error' due to https://github.com/microsoft/WSL/issues/8219#issuecomment-1133936081 — committed to ysoftwareab/yplatform by andreineculau 4 years ago
- Import Debian changes 1.10-4ubuntu4.1 gzip (1.10-4ubuntu4.1) jammy; urgency=medium * Cherry-pick upstream patch to use more portable alignment to resolve failure to execute on WSL1. https:... — committed to balabit-deps/balabit-os-9-gzip by xnox 4 years ago
Actually those were all 0x1000 already. In gzip 1.10-4ubuntu4 I only had to change the value at offset
0x189usingecho -en '\x10' | sudo dd of=/usr/bin/gzip count=1 bs=1 conv=notrunc seek=$((0x189))By reversing
lxcore.sys, the real reason of this bug is that WSL1 assumes allp_alignmember inPT_LOADprogram headers must be the same value, which is not correct. (See elf(5) — Linux manual page, there is not such assumption forp_align)Part of decompilation pesudo C code of
lxcore.sys(version10.0.22000.1from Windows 11 21H2 22000.556): lxcore.sys.zipLxpElfInfoParsefunction (RVA 0x1C004DC60) atlxcore.sysparses ELF file. See pesudo code above,0x464C457Fis the ELF magic number (“\x7fELF”), andv23isp_typemember ofElf64_Phdr.v23 == 1meansPT_LOAD(see here), andv54isp_alignmember ofElf64_Phdr.v46is initialized to0, and when it firstly meetsv54, it will be set to its value. Whenv46secondly meetsv54, it checksv54should equals to the oldv46value, which causes this issue.So, for example, after patching all the
p_alignvalue from0x1000to0x2000(offset0xE1,0x119and0x151of gzip1.10-4ubuntu4), the new binary can also works well.The old version gzip 1.10-4ubuntu1 works well, and the new version gzip 1.10-4ubuntu3 triggers this issue.
I compare their program headers by
readelf -l gzipcommand, and here are theLOADsegments: For 1.10-4ubuntu1 (which has no bug):and for 1.10-4ubuntu3 (which has the bug):
Notice that the buggy version has a strange
LOADsegment withAlign0x2000, and after patching the0x2000to0x1000(by modifying only one byte ofgzipbinary at offset0x189from0x20to0x10), the bug disappears and the patched binary works well!So, maybe WSL1 makes a wrong assumption that the
p_alignvalue is 0x1000. It is just a bug in WSL1 rather than Ubuntu, and seems quite easy to fix.I came here from #8151, nodejs from Arch could not run on WSL1.
In my case,
nodealso has different p_align values in its program headers, but this only happens if nodejs is built after 2022/02/14, when Archlinux upgraded glibc from 2.33 to 2.35. After some git bisect, I find out that GNU ld changed p_align in binutils 2.38, or more specifically binutils-gdb@74e315dbfe5. Previously, if a section requires alignment higher than max-page-size, it won’t affect p_align in corresponding segment. After this commit, if a section requires alignment higher than max-page-size, the required alignment will be set as p_align of this and later segments. In nodejs’s case, it has alpstubsection that is aligned to 2MiB, which caused several LOAD segments having p_align=0x200000.Here’s the python script I used to patch p_align values:
Microsoft thinks WSL 1 is dead )) Their evangelists use WSL 2 exclusively.
Is it possible to update lxcore.sys to fix the bug?
Same issue here (Ubuntu 22.04 & WSL1). I can’t start VS Code Server due this issue:
Older version works well:
Also a workaround without actually patching the binary:
From this comment on the related Node issue.
@rattfieldnz This issue is only for WSL 1!!!
@all
Another workaround is to run via
ld.so:for example the “fix” for
node:work!
A new
gzippackage is available in thejammy-proposedrepo that appears to fix the issue (well, at least revert the optimizations that cause WSL1 to choke). If you have the ability, please follow the instructions in the Launchpad report to test and report your findings.My understanding is that only one person needs to confirm, but I’d love more eyes on it than just mine.
has anyone bumped it to windows division? (I havent used windows in ages so i dont really know how) This really ought to be fixed as their linker is faulty and in future will continue to cause problems with newer builds
Can confirm this works for me as well.
My build info:
OS Name: Microsoft Windows 10 Home
Version: 10.0.19044 Build 19044
System Type: x64-based PC
WSL Version: 2
Linux Distro: Ubuntu 22.04 (Jammy)
It’s magic
@Zenderable OMG, you’re right, my bad… I have two Windows 11 computers, both with Windows Insiders, both with WSL, and I was sure both of them used the same WSL version… And not, this one has WSL 1, and the other WSL 2 (and it hasn’t the problem, you’re right). Sorry about that!
@LostInBrittany Are you absolutely certain that your distribution is running under WSL2? You can list all distros with
wsl -l -v. I ask, because when switching to WSL2 it didn’t happen to me. I tried patch by @dreamlayers on newest gzip and it’s working as it should!