podman: Error: initial journal cursor: failed to get cursor: cannot assign requested address

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

While testing new F35-beta VM images in https://github.com/containers/podman/pull/11795 multiple containerized-tests are failing with this error message. All of them involve the ‘podman logs’ command.

Steps to reproduce the issue:

  1. git checkout upstream/pr/11795
  2. hack/get_ci_vm.sh int remote fedora-35 root container
  3. podman run -it --rm --hostname test --privileged --net=host --cgroupns=host -v /dev/fuse:/dev/fuse -v "$GOPATH:$GOPATH:Z" --workdir "$GOSRC" $CTR_FQIN
  4. mkdir /tmp/test
  5. bin/podman --events-backend file create --name test --mount type=bind,src=/tmp/test,target=/myvol,z quay.io/libpod/alpine:latest touch /myvol/foo.txt
  6. bin/podman start test
  7. bin/podman logs test

Edit/Clarification: Reproducer steps 3-6 are done inside the container created in step 2.

Describe the results you received:

Error: initial journal cursor: failed to get cursor: cannot assign requested address

Describe the results you expected:

No output

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      4.0.0-dev
API Version:  4.0.0-dev
Go Version:   go1.16.8
Git Commit:   dd6de5b1f2f32c0b86b190886441d1ac7390d0c1
Built:        Mon Oct 18 20:46:19 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 2
  distribution:
    distribution: fedora
    variant: container
    version: "35"
  eventLogger: file
  hostname: test
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.14.12-300.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 839319552
  memTotal: 4104306688
  ociRuntime:
    name: crun
    package: crun-1.2-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.2
      commit: 4f6c8e0583c679bfee6a899c05ac6b916022561b
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 4104122368
  swapTotal: 4104122368
  uptime: 25m 15.07s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: vfs
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus: {}
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.0.0-dev
  Built: 1634589979
  BuiltTime: Mon Oct 18 20:46:19 2021
  GitCommit: dd6de5b1f2f32c0b86b190886441d1ac7390d0c1
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 4.0.0-dev

Package info (e.g. output of rpm -q podman or apt list podman):

Compiled from source

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

GCP VM Image fedora-c5160336597712896

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 42 (13 by maintainers)

Commits related to this issue

Most upvoted comments

@godmar, you can create following file at /etc/containers/containers.conf:

[engine]
log_driver="k8s-file"

Or edit /usr/share/containers/containers.conf (or copy that over to /etc/ and edit it there).

Thanks, setting log_driver="k8s-file" in /usr/share/containers/containers.conf resolved the issue, with the following caveats:

  • This setting applies only to new containers started after the setting took effect and not retroactively to existing containers, not even when they are stopped + restarted.
  • /usr/share/containers/containers.conf does not appear to have [engine] section.