podman: rhel78: mountpoint for devices not found

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

podman run with a non-root user does not work. The operating system is rhel78. slirp4netns and podman are installed by yum. Also user.max_user_namespaces=28633 is configured on the system.

Steps to reproduce the issue:

  1. run the following command with a non-root user.
podman run --rm -i -t busybox echo hello

Describe the results you received:

Error: container_linux.go:349: starting container process caused "process_linux.go:297: applying cgroup configuration for process caused \"mountpoint for devices not found\"": OCI runtime error

Describe the results you expected:

hello

Additional information you deem important (e.g. issue happens only occasionally):

It works with root user. Also, runc works with a non-root user. The following is a part of debug log of the podman:

time="2020-07-16T02:07:37-05:00" level=debug msg="Received: -1"
time="2020-07-16T02:07:37-05:00" level=debug msg="Cleaning up container 94adf317c678e7500e59cd943089b79636acb63a9e07897b4511262ab9d3342d"
time="2020-07-16T02:07:37-05:00" level=debug msg="Tearing down network namespace at /run/user/2050/netns/cni-cd27ba28-5a12-05fb-ae23-a4e7aaef8fef for container 94adf317c678e7500e59cd943089b79636acb63a9e07897b4511262ab9d3342d"
time="2020-07-16T02:07:37-05:00" level=debug msg="Error unmounting /home/user1/.local/share/containers/storage/overlay/9fbdc45f91a461d4cb1a7e4bbebe93defa88efbacf533ef5c415aee948dae09c/merged with fusermount3 - exec: \"fusermount3\": executable file not found in $PATH"
time="2020-07-16T02:07:37-05:00" level=debug msg="unmounted container \"94adf317c678e7500e59cd943089b79636acb63a9e07897b4511262ab9d3342d\""
time="2020-07-16T02:07:37-05:00" level=debug msg="ExitCode msg: \"time=\\\"2020-07-16t02:07:37-05:00\\\" level=warning msg=\\\"signal: killed\\\"\\ntime=\\\"2020-07-16t02:07:37-05:00\\\" level=error msg=\\\"container_linux.go:349: starting container process caused \\\\\\\"process_linux.go:297: applying cgroup configuration for process caused \\\\\\\\\\\\\\\"mountpoint for devices not found\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\ncontainer_linux.go:349: starting container process caused \\\"process_linux.go:297: applying cgroup configuration for process caused \\\\\\\"mountpoint for devices not found\\\\\\\"\\\": oci runtime error\""
time="2020-07-16T02:07:37-05:00" level=error msg="time=\"2020-07-16T02:07:37-05:00\" level=warning msg=\"signal: killed\"\ntime=\"2020-07-16T02:07:37-05:00\" level=error msg=\"container_linux.go:349: starting container process caused \\\"process_linux.go:297: applying cgroup configuration for process caused \\\\\\\"mountpoint for devices not found\\\\\\\"\\\"\"\ncontainer_linux.go:349: starting container process caused \"process_linux.go:297: applying cgroup configuration for process caused \\\"mountpoint for devices not found\\\"\": OCI runtime error"

Output of podman version:

Version:            1.6.4
RemoteAPI Version:  1
Go Version:         go1.12.12
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  git commit: ""
  go version: go1.12.12
  podman version: 1.6.4
host:
  BuildahVersion: 1.12.0-dev
  CgroupVersion: v1
  Conmon:
    package: conmon-2.0.15-1.el7_8.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.15, commit: 372b4a12f1c2df4f70c280d41173b60acd3f1260'
  Distribution:
    distribution: '"rhel"'
    version: "7.8"
  IDMappings:
    gidmap:
    - container_id: 0
      host_id: 2050
      size: 1
    - container_id: 1
      host_id: 558752
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 2050
      size: 1
    - container_id: 1
      host_id: 558752
      size: 65536
  MemFree: 16207659008
  MemTotal: 67250630656
  OCIRuntime:
    name: runc
    package: runc-1.0.0-67.rc10.el7_8.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.1-dev'
  SwapFree: 7665086464
  SwapTotal: 7665086464
  arch: amd64
  cpus: 8
  eventlogger: file
  hostname: tds-sbc2-el
  kernel: 3.10.0-1127.10.1.el7.x86_64
  os: linux
  rootless: true
  slirp4netns:
    Executable: /usr/bin/slirp4netns
    Package: slirp4netns-0.4.3-4.el7_8.x86_64
    Version: |-
      slirp4netns version 0.4.3
      commit: 2244b9b6461afeccad1678fac3d6e478c28b4ad6
  uptime: 910h 24m 25.53s (Approximately 37.92 days)
registries:
  blocked: null
  insecure: null
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  ConfigFile: /home/user1/.config/containers/storage.conf
  ContainerStore:
    number: 10
  GraphDriverName: overlay
  GraphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-0.7.2-6.el7_8.x86_64
      Version: |-
        fuse-overlayfs: version 0.7.2
        FUSE library version 3.6.1
        using FUSE kernel interface version 7.29
  GraphRoot: /home/user1/.local/share/containers/storage
  GraphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 3
  RunRoot: /run/user/2050/containers
  VolumePath: /home/user1/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.6.4-18.el7_8.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.): physical server

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

@a-trout-in-the-milk thanks for confirming it.

I don’t think such edge cases are going to be addressed in RHEL 7 anyway.

Feel free to include me in any Red Hat discussion on the problem you are having, but let’s close the issue here as it cannot be addressed anyway in future versions of Podman. For cgroup v2, we are already assuming all over the stack that cgroups are mounted at /sys/fs/cgroup