podman: Unable to start toolbox containers in Fedora Silverblue 31
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
Unable to start fedora-toolbox-31 (or any other toolbox container) in Fedora Silverblue 31.
Steps to reproduce the issue:
-
toolbox createto create a new toolbox on Fedora Silverblue 31 -
toolbox enterto enter the toolbox. This generates an error:toolbox: failed to start container fedora-toolbox-31 -
podman --log-level debug start fedora-toolbox-31to debug the issue generates a warning (see description)`
Describe the results you received:
The following output is presented when debuging with:
podman --log-level debug start fedora-toolbox-31
INFO[0000] running as rootless
DEBU[0000] using conmon: "/usr/libexec/podman/conmon"
DEBU[0000] Initializing boltdb state at /var/home/returntrip/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /var/home/returntrip/.local/share/containers/storage
DEBU[0000] Using run root /run/user/1000
DEBU[0000] Using static dir /var/home/returntrip/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /run/user/1000/libpod/tmp
DEBU[0000] Using volume path /var/home/returntrip/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: mount_program=/usr/bin/fuse-overlayfs
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false
DEBU[0000] Initializing event backend journald
DEBU[0000] using runtime "/usr/bin/crun"
DEBU[0000] using runtime "/usr/bin/runc"
DEBU[0000] overlay: mount_data=lowerdir=/var/home/returntrip/.local/share/containers/storage/overlay/l/HDZBVTBU4P5YJODD2W5VAHROK6:/var/home/returntrip/.local/share/containers/storage/overlay/l/UNHS2OY3XUUJNRAEMMDBKQJA23,upperdir=/var/home/returntrip/.local/share/containers/storage/overlay/03373eec54ee786f0ae0abee768212b4bd5122d531fbe298d9713b1b44e723f6/diff,workdir=/var/home/returntrip/.local/share/containers/storage/overlay/03373eec54ee786f0ae0abee768212b4bd5122d531fbe298d9713b1b44e723f6/work,context="system_u:object_r:container_file_t:s0:c687,c748"
DEBU[0000] mounted container "2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c" at "/var/home/returntrip/.local/share/containers/storage/overlay/03373eec54ee786f0ae0abee768212b4bd5122d531fbe298d9713b1b44e723f6/merged"
DEBU[0000] Created root filesystem for container 2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c at /var/home/returntrip/.local/share/containers/storage/overlay/03373eec54ee786f0ae0abee768212b4bd5122d531fbe298d9713b1b44e723f6/merged
DEBU[0000] /etc/system-fips does not exist on host, not mounting FIPS mode secret
DEBU[0000] Setting CGroups for container 2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c to user.slice:libpod:2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c
DEBU[0000] set root propagation to "rslave"
DEBU[0000] Created OCI spec for container 2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c at /var/home/returntrip/.local/share/containers/storage/overlay-containers/2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c/userdata/config.json
DEBU[0000] /usr/libexec/podman/conmon messages will be logged to syslog
DEBU[0000] running conmon: /usr/libexec/podman/conmon args="[--api-version 1 -s -c 2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c -u 2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c -r /usr/bin/crun -b /var/home/returntrip/.local/share/containers/storage/overlay-containers/2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c/userdata -p /run/user/1000/overlay-containers/2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c/userdata/pidfile -l k8s-file:/var/home/returntrip/.local/share/containers/storage/overlay-containers/2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c/userdata/ctr.log --exit-dir /run/user/1000/libpod/tmp/exits --socket-dir-path /run/user/1000/libpod/tmp/socket --log-level debug --syslog --conmon-pidfile /run/user/1000/overlay-containers/2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/home/returntrip/.local/share/containers/storage --exit-command-arg --runroot --exit-command-arg /run/user/1000 --exit-command-arg --log-level --exit-command-arg error --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/user/1000/libpod/tmp --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mount_program=/usr/bin/fuse-overlayfs --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg 2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c]"
INFO[0000] Running conmon under slice user.slice and unitName libpod-conmon-2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c.scope
WARN[0000] Failed to add conmon to systemd sandbox cgroup: write unix @: sendmsg: broken pipe
[conmon:d]: failed to write to /proc/self/oom_score_adj: Permission denied
DEBU[0000] Cleaning up container 2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c
DEBU[0000] Network is already cleaned up, skipping...
DEBU[0000] unmounted container "2083b2d1cc90d630152ae3a61e0f9cd68266ce985e5ff282f7e54811183b065c"
ERRO[0000] unable to start container "fedora-toolbox-31": error reading container (probably exited) json message: EOF
Describe the results you expected:
User should be able to access toolbox without errors
Additional information you deem important (e.g. issue happens only occasionally): Discussed with Rishi and he suggested to file an issue here
Output of podman version:
podman version 1.5.1-dev
Output of podman info --debug:
debug:
compiler: gc
git commit: ""
go version: go1.13rc1
podman version: 1.5.1-dev
host:
BuildahVersion: 1.10.1
Conmon:
package: podman-1.5.1-2.17.dev.gitce64c14.fc31.x86_64
path: /usr/libexec/podman/conmon
version: 'conmon version 2.0.0, commit: 118fcdfca36d706f766bad2663b11bd2c41bf2e7'
Distribution:
distribution: fedora
version: "31"
MemFree: 7146295296
MemTotal: 16778080256
OCIRuntime:
package: crun-0.8-1.fc31.x86_64
path: /usr/bin/crun
version: |-
crun 0.8
spec: 1.0.0
+SYSTEMD +SELINUX +CAP +SECCOMP +EBPF +YAJL
SwapFree: 7985950720
SwapTotal: 7985950720
arch: amd64
cpus: 16
eventlogger: journald
hostname: rauros.figura.io
kernel: 5.3.0-0.rc6.git0.1.fc31.x86_64
os: linux
rootless: true
uptime: 1h 55m 38.6s (Approximately 0.04 days)
registries:
blocked: null
insecure: null
search:
- docker.io
- registry.fedoraproject.org
- quay.io
- registry.access.redhat.com
- registry.centos.org
store:
ConfigFile: /home/returntrip/.config/containers/storage.conf
ContainerStore:
number: 1
GraphDriverName: overlay
GraphOptions:
- overlay.mount_program=/usr/bin/fuse-overlayfs
GraphRoot: /var/home/returntrip/.local/share/containers/storage
GraphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "false"
Supports d_type: "true"
Using metacopy: "false"
ImageStore:
number: 1
RunRoot: /run/user/1000
VolumePath: /var/home/returntrip/.local/share/containers/storage/volumes
Package info (e.g. output of rpm -q podman or apt list podman):
podman-1.5.1-2.17.dev.gitce64c14.fc31.x86_64
Additional environment details (AWS, VirtualBox, physical, etc.): Phisical machine. Rebased FSB30 to FSB31
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 36 (25 by maintainers)
made a new release: https://bodhi.fedoraproject.org/updates/FEDORA-2019-4b4957bbc6
I too am seeing this issue, and after applying overrides for podman and crun, and unsetting NOTIFY_SOCKET, containers still fail to start.
Versions:
podman 1.6.0-2.gitca5ff03.fc31crun 0.9.1-1.fc31Logs: