podman: Rootless container exits immediately when run via a GitHub action but runs fine locally
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
When running a rootless container with Podman on Ubuntu-20.04 via GitHub actions the container immediately quits.
Starting the same container image with sudo podman ... (rootless: false) works fine via GitHub actions, as does running the container with Docker.
The same image runs fine (rootless: true) with an equivalent install (as close as possible) of Podman on an Ubuntu-20.04 VM.
Steps to reproduce the issue:
The issue can be consistently reproduced.
-
Fork the GitHub repository that demonstrates the issue HERE.
-
Go to the
Actionstab -
Click on the
Demo issueworkflow. -
Click on the
Run workflowdrop down on the right hand side of the screen and then clickRun workflow.
Describe the results you received:
Instead of continuing to run in detached mode the container immediately quits - the STATUS field in the output of podman ps -a shows Exited (255)....
Describe the results you expected:
The container should continue to run in detached mode - the same way it does on my local system and in my Ubuntu 20.04 VM.
Additional information you deem important (e.g. issue happens only occasionally):
- The container is running systemd as PID 1 i.e. the entrypoint is configured to run
/lib/systemd/systemd. - I have tried running Podman with
--systemd=always- this has no effect. - The
[conmon:d]: failed to write to /proc/self/oom_score_adj: Permission deniederror (seen in the output ofpodman run --log-level=debug...) appears to be a red herring - I see the same in the logs when containers run successfully as well. - I have tried installing
libpam-cgfsin the GitHub actions Ubuntu VM - this has no effect. - I have tried running Podman with
--volume /sys/fs/cgroup:/sys/fs/cgroup:ro- this has no effect.
Output of podman version:
From the GitHub Action workflow debug output:
Version: 3.1.2
API Version: 3.1.2
Go Version: go1.15.2
Built: Thu Jan 1 00:00:00 1970
OS/Arch: linux/amd64
Output of podman info --debug:
From the GitHub Action workflow debug output:
host:
arch: amd64
buildahVersion: 1.20.1
cgroupManager: cgroupfs
cgroupVersion: v1
conmon:
package: 'conmon: /usr/libexec/podman/conmon'
path: /usr/libexec/podman/conmon
version: 'conmon version 2.0.27, commit: '
cpus: 2
distribution:
distribution: ubuntu
version: "20.04"
eventLogger: journald
hostname: fv-az93-395
idMappings:
gidmap:
- container_id: 0
host_id: 121
size: 1
- container_id: 1
host_id: 165536
size: 65536
uidmap:
- container_id: 0
host_id: 1001
size: 1
- container_id: 1
host_id: 165536
size: 65536
kernel: 5.4.0-1047-azure
linkmode: dynamic
memFree: 4737449984
memTotal: 7292141568
ociRuntime:
name: crun
package: 'crun: /usr/bin/crun'
path: /usr/bin/crun
version: |-
crun version 0.19.1.3-9b83-dirty
commit: 33851ada2cc9bf3945915565bf3c2df97facb92c
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
os: linux
remoteSocket:
path: /tmp/podman-run-1001/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
selinuxEnabled: false
slirp4netns:
executable: /usr/bin/slirp4netns
package: 'slirp4netns: /usr/bin/slirp4netns'
version: |-
slirp4netns version 1.1.8
commit: unknown
libslirp: 4.3.1-git
SLIRP_CONFIG_VERSION_MAX: 3
libseccomp: 2.4.3
swapFree: 4294963200
swapTotal: 4294963200
uptime: 4m 43.88s
registries:
search:
- docker.io
- quay.io
store:
configFile: /home/runner/.config/containers/storage.conf
containerStore:
number: 0
paused: 0
running: 0
stopped: 0
graphDriverName: overlay
graphOptions:
overlay.mount_program:
Executable: /usr/bin/fuse-overlayfs
Package: 'fuse-overlayfs: /usr/bin/fuse-overlayfs'
Version: |-
fusermount3 version: 3.9.0
fuse-overlayfs: version 1.5
FUSE library version 3.9.0
using FUSE kernel interface version 7.31
graphRoot: /home/runner/.local/share/containers/storage
graphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "false"
Supports d_type: "true"
Using metacopy: "false"
imageStore:
number: 3
runRoot: /tmp/podman-run-1001/containers
volumePath: /home/runner/.local/share/containers/storage/volumes
version:
APIVersion: 3.1.2
Built: 0
BuiltTime: Thu Jan 1 00:00:00 1970
GitCommit: ""
GoVersion: go1.15.2
OsArch: linux/amd64
Version: 3.1.2
Side by side diff of podman info --debug from Ubuntu-20.04 running via GitHub actions (left) and from Ubuntu-20.04 VM - shows differences in idMappings.
Could this be causing the issue?
host: (
arch: amd64 (
buildahVersion: 1.20.1 (
cgroupManager: cgroupfs (
cgroupVersion: v1 (
conmon: (
package: 'conmon: /usr/libexec/podman/conmon' (
path: /usr/libexec/podman/conmon (
version: 'conmon version 2.0.27, commit: ' (
cpus: 2 | cpus: 1
distribution: (
distribution: ubuntu (
version: "20.04" (
eventLogger: journald (
hostname: fv-az93-395 | hostname: focal
idMappings: (
gidmap: (
- container_id: 0 (
host_id: 121 | host_id: 1000
size: 1 (
- container_id: 1 (
host_id: 165536 | host_id: 100000
size: 65536 (
uidmap: (
- container_id: 0 (
host_id: 1001 | host_id: 1000
size: 1 (
- container_id: 1 (
host_id: 165536 | host_id: 100000
size: 65536 (
kernel: 5.4.0-1047-azure | kernel: 5.4.0-73-generic
linkmode: dynamic (
memFree: 4737449984 | memFree: 716386304
memTotal: 7292141568 | memTotal: 2084315136
ociRuntime: (
name: crun (
package: 'crun: /usr/bin/crun' (
path: /usr/bin/crun (
version: |- (
crun version 0.19.1.3-9b83-dirty (
commit: 33851ada2cc9bf3945915565bf3c2df97facb92c (
spec: 1.0.0 (
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL (
os: linux (
remoteSocket: (
path: /tmp/podman-run-1001/podman/podman.sock | path: /run/user/1000/podman/podman.sock
security: (
apparmorEnabled: false (
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KI (
rootless: true (
seccompEnabled: true (
selinuxEnabled: false (
slirp4netns: (
executable: /usr/bin/slirp4netns (
package: 'slirp4netns: /usr/bin/slirp4netns' (
version: |- (
slirp4netns version 1.1.8 (
commit: unknown (
libslirp: 4.3.1-git (
SLIRP_CONFIG_VERSION_MAX: 3 (
libseccomp: 2.4.3 (
swapFree: 4294963200 | swapFree: 0
swapTotal: 4294963200 | swapTotal: 0
uptime: 4m 43.88s | uptime: 8h 38m 4.59s (Approximately 0.33 days)
registries: (
search: (
- docker.io (
- quay.io (
store: (
configFile: /home/runner/.config/containers/storage.conf | configFile: /home/vagrant/.config/containers/storage.conf
containerStore: (
number: 0 (
paused: 0 (
running: 0 (
stopped: 0 (
graphDriverName: overlay (
graphOptions: (
overlay.mount_program: (
Executable: /usr/bin/fuse-overlayfs (
Package: 'fuse-overlayfs: /usr/bin/fuse-overlayfs' (
Version: |- (
fusermount3 version: 3.9.0 (
fuse-overlayfs: version 1.5 (
FUSE library version 3.9.0 (
using FUSE kernel interface version 7.31 (
graphRoot: /home/runner/.local/share/containers/storage | graphRoot: /home/vagrant/.local/share/containers/storage
graphStatus: (
Backing Filesystem: extfs (
Native Overlay Diff: "false" (
Supports d_type: "true" (
Using metacopy: "false" (
imageStore: (
number: 3 (
runRoot: /tmp/podman-run-1001/containers | runRoot: /run/user/1000/containers
volumePath: /home/runner/.local/share/containers/storage/volumes | volumePath: /home/vagrant/.local/share/containers/storage/volumes
version: (
APIVersion: 3.1.2 (
Built: 0 (
BuiltTime: Thu Jan 1 00:00:00 1970 (
GitCommit: "" (
GoVersion: go1.15.2 (
OsArch: linux/amd64 (
Version: 3.1.2 (
Package info (e.g. output of rpm -q podman or apt list podman):
From the GitHub Action workflow debug output:
Listing...
podman/now 100:3.1.2-1 amd64 [installed,local]
Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)
Yes
Additional environment details (AWS, VirtualBox, physical, etc.):
See the workflow file and debug output in the workflow run.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 26 (14 by maintainers)
I don’t see an issue with it, other then podentially allowing the id user to chown the content, but that user already is allowed sudo, so I really don’t see this as a problem.
Thanks, @DanHam. It also runs fine on my local system. @giuseppe is the cgroups experts, and I am sure he knows what’s going on. It could be that install dbus-launch will solve the problem but I wonder why we’re not hitting the issue as root.
The results of the previous runs just showed up now. Looks like it takes a while; maybe related to the recent GitHub outage. Thanks 😃
There should be a ‘Run workflow’ drop down/button on the right hand side. Click that. Then click the green ‘Run Workflow’ button
If you can’t see that ‘Run workflow’ drop down then you may be on the wrong page. So:
Thanks, @DanHam! Yes, I saw the reproducers but am short on time juggling a number of issues in parallel at the moment. I may find more time tomorrow to look into it.