podman: new error (used to work before) on podman run invocation: Cannot get exit code: failed to get journal cursor: failed to get cursor: cannot assign requested address

Description

Recently I’ve been getting the following error on a CI that used to run without problems. The CI runner is on bare metal (Manjaro) Linux, and may have been updated in the meantime, but I can’t be sure this error is related to an update of my podman version (currently: 3.2.2).

ERRO[0002] Cannot get exit code: failed to get journal cursor: failed to get cursor: cannot assign requested address

when executing (“call-graph-like” representation):

Makefile -> podman run -> script.sh -> exec go build …

I figured it might be caused in some way by the podman log-driver, but using

podman run --log-driver=none ...

doesn’t have any effect.

I’m at a loss what might be causing this or how to debug this issue…

Output of podman version:

podman version 3.2.2

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.21.0
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: /usr/bin/conmon is owned by conmon 1:2.0.29-1
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: 7e6de6678f6ed8a18661e1d5721b81ccee293b9b'
  cpus: 24
  distribution:
    distribution: manjaro
    version: unknown
  eventLogger: journald
  hostname: jmanji
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 107
      size: 1
    - container_id: 1
      host_id: 362144
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 107
      size: 1
    - container_id: 1
      host_id: 362144
      size: 65536
  kernel: 5.11.4-1-rt11-MANJARO
  linkmode: dynamic
  memFree: 14399438848
  memTotal: 67370762240
  ociRuntime:
    name: crun
    package: /usr/bin/crun is owned by crun 0.20.1-2
    path: /usr/bin/crun
    version: |-
      crun version 0.20.1
      commit: 38271d1c8d9641a2cdc70acfa3dcb6996d124b3d
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/107/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: /usr/bin/slirp4netns is owned by slirp4netns 1.1.11-1
    version: |-
      slirp4netns version 1.1.11
      commit: 368e69ccc074628d17a9bb9a35b8f4b9f74db4c6
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.1
  swapFree: 30744055808
  swapTotal: 32133943296
  uptime: 549h 28m 41.2s (Approximately 22.88 days)
registries:
  1nnoserv:15000:
    Blocked: false
    Insecure: true
    Location: 1nnoserv:15000
    MirrorByDigestOnly: false
    Mirrors: []
    Prefix: 1nnoserv:15000
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  configFile: /home/gitlab-runner/.config/containers/storage.conf
  containerStore:
    number: 28
    paused: 0
    running: 0
    stopped: 28
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: /usr/bin/fuse-overlayfs is owned by fuse-overlayfs 1.6-1
      Version: |-
        fusermount3 version: 3.10.4
        fuse-overlayfs: version 1.6
        FUSE library version 3.10.4
        using FUSE kernel interface version 7.31
  graphRoot: /home/gitlab-runner/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 16
  runRoot: /run/user/107/containers
  volumePath: /home/gitlab-runner/.local/share/containers/storage/volumes
version:
  APIVersion: 3.2.2
  Built: 1625835244
  BuiltTime: Fri Jul  9 14:54:04 2021
  GitCommit: d577c44e359f9f8284b38cf984f939b3020badc3
  GoVersion: go1.16.5
  OsArch: linux/amd64
  Version: 3.2.2

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

Hello there, I experienced the same error but in a different situation (upgrading Fedora IoT from 34 to 35): suddenly, I got the same error that @ppenguin reported in the issue. However, creating the container.conf file under the home directory of the user I ran the containers with, and setting log_driver = "k8s-file" and de-commenting events_logger = "journald", I could then get the logs again with the podman logs command, without putting my unprivileged user into the group wheel. Notably, none of the other 3 combinations for the 2 options I mentioned above, lead to the unprivileged user reading the logs.

cp /usr/share/containers/containers.conf /etc/containers/containers.conf
or
cp /usr/share/containers/containers.conf $HOME/.config/containers/containers.conf

$ grep events /usr/share/containers/containers.conf 
# Selects which logging mechanism to use for container engine events.
# events_logger = "journald"

Uncomment the events_logger line and change it to “file”.

man containers.conf