lxc-ci: Ubuntu 23.10 (mantic) containers fail to start

Starting a freshly created mantic (20231014_07:42) container results in:

# lxc-create -n foobar -t download -- -d ubuntu -r mantic -a amd64
Downloading the image index
Downloading the rootfs
Downloading the metadata
The image cache is now ready
Unpacking the rootfs

---
You just created an Ubuntu mantic amd64 (20231014_07:42) container.

To enable SSH, run: apt install openssh-server
No default root or user password are set by LXC.

# lxc-start -n foobar -F
systemd 253.5-1ubuntu6 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
Detected virtualization lxc.
Detected architecture x86-64.

Welcome to Ubuntu 23.10!

Initializing machine ID from random generator.
Failed to fork off sandboxing environment for executing generators: Protocol error
[!!!!!!] Failed to start up manager.
Exiting PID 1...

This looks very much like systemd/systemd#27436, so it might indeed be due to some issue with /tmp not being available in the initial file system. I will check if I can work around it with a custom config file.

About this issue

Original URL
State: closed
Created 9 months ago
Comments: 16 (7 by maintainers)

Most upvoted comments

Ah, right so you’re hitting:

[5271036.661234] audit: type=1400 audit(1697561450.860:160): apparmor="DENIED" operation="mount" info="failed flags match" e
rror=-13 profile="lxc-container-default-cgns" name="/" pid=1176445 comm="(sd-gens)" flags="rw, rslave"

This is something that’s been fixed in https://github.com/lxc/lxc/pull/4295 but hasn’t been part of an LXC release yet as we’re waiting on the various distros to catch the newer apparmor first.

If your system has apparmor 2.13.8 or 3.0.10 (and newer), then you could try applying the changes directly to your files in /etc/apparmor.d, reboot the system and try again: https://github.com/lxc/lxc/pull/4295/files

That should take care of the DENIED that you’re seeing in dmesg and hopefully let systemd boot properly.

This whole mess came from a bad security bug in apparmor which apparmor upstream pretty much ignored for a decade until @mihalicyn decided to just go and fix it for them earlier this year. We’ve been waiting for the fix to trickle into all distros before we get to relax our policies without causing major security issues (allowing the container to take over the whole system).

stgraber on Oct 17, 2023