podman: "cgroup is not set: internal libpod error" when adding container to existing pod while rootless

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

With podman v3.2.x, when running rootless on Alpine Linux (no systemd, cgroups switched to v2/“unified”), adding container to a existing pod that was stopped and started at least once, started showing errors like this:

Error: pod ec578f75616cc98c4c449e5f6590e6bc8e9309ab1032d3def6d42a381ad17527 cgroup is not set: internal libpod error

Steps to reproduce the issue:

  1. Create pod with some container.

  2. Start the pod.

  3. Stop the pod.

  4. Try adding container (AFAIK does not matter if it’s “temporary”, i.e., with --rm, or not) to a pod:

podman run --pod testpod --rm docker.io/alpine sh -c "date"

Describe the results you received:

Error information about cgroup:

Error: pod ec578f75616cc98c4c449e5f6590e6bc8e9309ab1032d3def6d42a381ad17527 cgroup is not set: internal libpod error

Describe the results you expected:

No error, output returned and exit with status 0.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

podman version 3.2.2

Output of podman info --debug:

(paste your output here)

Package info (e.g. output of rpm -q podman or apt list podman):

host:
  arch: amd64
  buildahVersion: 1.21.0
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: b388b959974dee50d451f88949b3499c3ca6ca42'
  cpus: 1
  distribution:
    distribution: alpine
    version: 3.14.0
  eventLogger: file
  hostname: localhost
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 10000
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 10000
  kernel: 5.10.43-0-virt
  linkmode: dynamic
  memFree: 3857145856
  memTotal: 4134678528
  ociRuntime:
    name: crun
    package: Unknown
    path: /usr/bin/crun
    version: |-
      crun version 0.20.1
      commit: 38271d1c8d9641a2cdc70acfa3dcb6996d124b3d
      spec: 1.0.0
      +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /tmp/podman-run-1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: Unknown
    version: |-
      slirp4netns version 1.1.10
      commit: baa2bc5ff12fe6db646c1f4f3f966526c0eba5a0
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.1
  swapFree: 1342173184
  swapTotal: 1342173184
  uptime: 32m 42.89s
registries:
  search:
  - docker.io
store:
  configFile: /home/me/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: Unknown
      Version: |-
        fuse-overlayfs: version 1.6
        fusermount3 version: 3.10.4
        FUSE library version 3.10.4
        using FUSE kernel interface version 7.31
  graphRoot: /home/me/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 9
  runRoot: /tmp/podman-run-1000/containers
  volumePath: /home/me/.local/share/containers/storage/volumes
version:
  APIVersion: 3.2.2
  Built: 1624674245
  BuiltTime: Sat Jun 26 04:24:05 2021
  GitCommit: 94b97c166e51039997c5fd0658793af2cff0cb06
  GoVersion: go1.16.5
  OsArch: linux/amd64
  Version: 3.2.2

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Podman is run on Alpine Linux within QEmu (KVM). There is no systemd there, only OpenRC. Cgroups switched from v1 to v2 (v2 only, not a “mix of both” type of thing).

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 30 (15 by maintainers)

Commits related to this issue

Most upvoted comments

No, I think we’re set. I’m presently working on related changes to pod cgroups. I will tackle this once I am done with those.

@mheon @giuseppe any progress on pod cgroup fix?

I’ll take this one. Vague suspicion it is fixed in main with the patches from @cdoern to change cgroup parent for pods to the infra container’s cgroup.