WSL: `systemd status` fails output on Ubuntu
Version
10.0.22621.521
WSL Version
- WSL 2
- WSL 1
Kernel Version
5.15.62.1
Distro Version
Ubuntu 22.10
Other Software
No response
Repro Steps
install Ubuntu (Preview), store id 9P7BDVKVNXZ6. This version has systemd already enabled.
Expected Behavior
systemctl status command to work
Actual Behavior
toor@XPS:~$ systemctl status
Failed to dump process list for 'XPS', ignoring: Input/output error
● XPS
State: degraded
Units: 318 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 1 units
Since: Sat 2022-09-24 08:43:15 PDT; 14min ago
systemd: 251.4-1ubuntu6
Tainted: cgroupsv1
CGroup: /
It doesn’t work on ArchLinux also:
[toor@XPS ~]$ systemctl status
Failed to dump process list for 'XPS', ignoring: Input/output error
● XPS
State: running
Units: 243 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 0 units
Since: Sat 2022-09-24 08:51:04 PDT; 14s ago
systemd: 251.4-1-arch
Tainted: cgroupsv1
CGroup: /
Both WSL instances run systemd-251.4, so this could be a systemd issue. It works okay on Ubuntu 22.04 and systemd-249.11.
Diagnostic Logs
No response
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 13
- Comments: 40 (1 by maintainers)
Commits related to this issue
- cgroup-util: allow cg_read_pid() to return unmappable PIDs In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from o... — committed to keszybz/systemd by keszybz 3 years ago
- cgroup-util: allow cg_read_pid() to return unmappable PIDs In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from o... — committed to keszybz/systemd by keszybz 3 years ago
- cgroup-util: allow cg_read_pid() to return unmappable PIDs In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from o... — committed to keszybz/systemd by keszybz 3 years ago
- cgroup-util: allow cg_read_pid() to return unmappable PIDs In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from o... — committed to keszybz/systemd by keszybz 3 years ago
- cgroup-util: allow cg_read_pid() to return unmappable PIDs In some environments, namely WSL2, the cgroup.procs PID list for some reason contain a ton of zeros everywhere, most likely those are from o... — committed to keszybz/systemd by keszybz 3 years ago
It now works for me
I’ve applied the following workaround to my systemd on WSL, and it makes the output look like (I think) it’s supposed to look:
It’s most likely not a proper fix or a good idea to do, but it gets stuff working for me in the meantime.
@themasch the latest stable version of systemd 255 is 255.7 not 255.4. So you have to wait for FC to upgrade systemd.
This is fixed in the latest systemd stable version of 255, so someone just has to get Ubuntu to update.
This issue is reproducible on WSL-based Arch Linux with
systemd 254 (254.5-1-arch).Still an issue to date.
Also tested with pre-release 1.2.4. Both using cgroups v2. Of the Ubuntu apps, that situation still only affects the Ubuntu-Preview for now, but eventually other releases will be at systemd version 2.50 or later, so that will be a problem.
A suitable fix seems to be to create and assign a cgroup and a cgroup namespace to every distro instance. Handling the cgroup2 mount point properly before starting systemd, it should then see that as its root cgroup. There will be no PID 0’s inside the
cgroup.procsfile, because it will be populated only by the PIDs of the child processes of systemd. Also, care must be taken to ensure the 'login" shell also goes to the that cgroup (and namespaces).For the sake of demonstration, if instead of enabling systemd through the proper settings in
wsl.confwe use the hack described below, then we getsystemctl statusbehaving correctly, with the downside of the root cgroup being displayed not as/.DISCLAIMER: By no means the above is a solution. It has many drawbacks, such as not showing the systemd root cgroup as
/as well as requiring more hacks to run processes (such as a shell) under the systemd namespaces.Configuring the boot command parameter to run that script and starting a shell under systemd namespaces allows us to see the following
systemctloutput.Works and was dead easy to get into NixOS:
I guess this issue will get more attention over time with more distos update systemd to >= 250. At the moment it seems that the only supported distro with such an systemd version is Kali (but I did not verify this).
Still an issue also on the latest 1.0.0
I think it is 3; that’s the same root cause @nullpo-head figured out for the problem when it was reported under distrod (https://github.com/nullpo-head/wsl-distrod/issues/31#issuecomment-1079726209), with the same fix (put systemd in a new cgroup). Of course, with systemd now running as pid 1 without a hack, the fix’ll need to be in WSL proper.
I’ve done some investigating: This regression was introduced by systemd/systemd@038cae098b6d85ad9fb06594d36f49915d32f000, which comes from this PR systemd/systemd#21172. It’s possible this is related to systemd/systemd#22089 but I’m not sure as it’s slightly different symptoms and isn’t fixed on main. That commit was a change to
systemctl statusto use the systemd dbus instead of “directly via cgroupfs” (I’m not sure what that means). I can replicate this bug by calling the dbus directly withbusctl call -- org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager GetUnitProcesses 's' '-.slice'. Some code diving suggests that the cause of the IO error is that systemd is not expecting to see zero PIDs in/sys/fs/cgroup/unified/cgroup.procs, which I have a bunch of, I believe they’re there because systemd is in a PID namespace but not in a new cgroup.I see 3 possible places the bug might be:
systemctl statuscode: it possibly shouldn’t be making that dbus call in that way, however, I’m skeptical as that API sounds sensible from its name./sys/fs/cgroup/unified/cgroup.procs.