podman: rootless podman doesn't work via sudo/su

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Running with --pids-limit causes an error

Error: cannot set limits without cgroups: OCI runtime error

Note: this is not a dup of #6834 which is about cgroupsv1.

Steps to reproduce the issue:

1. podman run --pids-limit 3123 -it --rm docker.io/fedora:32 echo hey

Describe the results you received:

Error: cannot set limits without cgroups: OCI runtime error

Describe the results you expected:

hey

Additional information you deem important (e.g. issue happens only occasionally):

Happens on a particular machine. I did podman system migrate, then podman system reset, still happens.

Output of podman version:

podman version 2.0.4

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.15.0
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.19-1.fc32.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.19, commit: 5dce9767526ed27f177a8fa3f281889ad509fea7'
  cpus: 4
  distribution:
    distribution: fedora
    version: "32"
  eventLogger: file
  hostname: sif.cloudius-systems.com
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1001
      size: 1
    - container_id: 1
      host_id: 300000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1001
      size: 1
    - container_id: 1
      host_id: 300000
      size: 65536
  kernel: 5.7.11-200.fc32.x86_64
  linkmode: dynamic
  memFree: 23911043072
  memTotal: 33618317312
  ociRuntime:
    name: crun
    package: crun-0.14.1-1.fc32.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.14.1
      commit: 598ea5e192ca12d4f6378217d3ab1415efeddefa
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1001/podman/podman.sock
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.4-1.fc32.x86_64
    version: |-
      slirp4netns version 1.1.4
      commit: b66ffa8e262507e37fca689822d23430f3357fe8
      libslirp: 4.3.1
      SLIRP_CONFIG_VERSION_MAX: 2
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 153h 56m 54.03s (Approximately 6.38 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /jenkins/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.1.2-1.fc32.x86_64
      Version: |-
        fusermount3 version: 3.9.1
        fuse-overlayfs: version 1.1.0
        FUSE library version 3.9.1
        using FUSE kernel interface version 7.31
  graphRoot: /jenkins/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 4
  runRoot: /run/user/1001/containers
  volumePath: /jenkins/.local/share/containers/storage/volumes
version:
  APIVersion: 1
  Built: 0
  BuiltTime: Thu Jan  1 02:00:00 1970
  GitCommit: ""
  GoVersion: go1.14.6
  OsArch: linux/amd64
  Version: 2.0.4

Package info (e.g. output of rpm -q podman or apt list podman):

podman-2.0.4-1.fc32.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical, fully updated Fedora 32, happens on this one host.