podman: Regression in 4.4 and 4.5: podman login can no longer be used with docker-compose

Issue Description

All of the following relates to rootless podman. I have no setup to test with rootful podman

Up until at least podman-4.2.0-11.el9_1.x86_64 we were able to use Docker Compose version v2.17.3 to pull images from an authenticated registry such as docker.io (via podman login --authfile ~/.docker/config.json docker.io). Starting with at least podman-4.4.1-9.el9_2.x86_64, this is no longer possible. Neither with the authentication stored in ~/.docker/config.json nor with it stored in ~/.config/containers/auth.json (nor with it stored in both, via multiple calls to podman login) will the credentials be accessed/applied when images are pulled via docker-compose. Identical auth files and docker-compose versions were used for testing to narrow the problem down to a change in podman between the two versions listed above. podman pull itself correctly uses the credentials.

Steps to reproduce the issue

Steps to reproduce the issue

  1. podman login --authfile ~/.docker/config.json docker.io
  2. docker-compose pull <some service with an image definition like image: docker.io/library/redis:6.2.6>

Describe the results you received

Cry because you’ve hit the rate limits and your account credentials are not used for pulling

Describe the results you expected

The joy of successfully pulling the image with my provided credentials, as was the case with podman 4.2.

podman info output

Working:

host:
  arch: amd64
  buildahVersion: 1.27.3
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.4-1.el9.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.4, commit: fd49ef99363f06fe6b6ab119070cd95c6cc7c35a'
  cpuUtilization:
    idlePercent: 99.9
    systemPercent: 0.08
    userPercent: 0.02
  cpus: 6
  distribution:
    distribution: '"rocky"'
    version: "9.1"
  eventLogger: file
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1006
      size: 1
    - container_id: 1
      host_id: 427680
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1005
      size: 1
    - container_id: 1
      host_id: 427680
      size: 65536
  kernel: 5.14.0-162.23.1.el9_1.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 3159465984
  memTotal: 3831787520
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.5-1.el9.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.5
      commit: 54ebb8ca8bf7e6ddae2eb919f5b82d1d96863dea
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1005/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-2.el9.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 2147479552
  swapTotal: 2147479552
  uptime: 1h 34m 54.00s (Approximately 0.04 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/me/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/me/.local/share/containers/storage
  graphRootAllocated: 18238930944
  graphRootUsed: 2104717312
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/user/1005/containers
  volumePath: /home/me/.local/share/containers/storage/volumes
version:
  APIVersion: 4.2.0
  Built: 1677602055
  BuiltTime: Tue Feb 28 17:34:15 2023
  GitCommit: ""
  GoVersion: go1.18.9
  Os: linux
  OsArch: linux/amd64
  Version: 4.2.0

Broken:

host:
  arch: amd64
  buildahVersion: 1.29.0
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.7-1.el9_2.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.7, commit: e6cdc9a4d6319e039efa13e532c1e58b713c904d'
  cpuUtilization:
    idlePercent: 99.62
    systemPercent: 0.12
    userPercent: 0.26
  cpus: 12
  distribution:
    distribution: '"rocky"'
    version: "9.2"
  eventLogger: file
  hostname: gitlab.server
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1009
      size: 1
    - container_id: 1
      host_id: 624288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1008
      size: 1
    - container_id: 1
      host_id: 624288
      size: 65536
  kernel: 5.14.0-284.11.1.el9_2.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 41302462464
  memTotal: 42062581760
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.8.4-1.el9_2.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.4
      commit: 5a8fa99a5e41facba2eda4af12fa26313918805b
      rundir: /run/user/1008/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1008/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-3.el9.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 42948599808
  swapTotal: 42948599808
  uptime: 1h 8m 2.00s (Approximately 0.04 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/gitlab-server/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/gitlab-server/.local/share/containers/storage
  graphRootAllocated: 8578379776
  graphRootUsed: 223911936
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/user/1008/containers
  transientStore: false
  volumePath: /home/gitlab-server/.local/share/containers/storage/volumes
version:
  APIVersion: 4.4.1
  Built: 1683632637
  BuiltTime: Tue May  9 13:43:57 2023
  GitCommit: ""
  GoVersion: go1.19.6
  Os: linux
  OsArch: linux/amd64
  Version: 4.4.1

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

No

Additional environment details

Behaviour tried in different virtual machines (kvm, virtualbox), always reproducible

Additional information

I am aware of https://github.com/containers/podman/issues/15620 and the solution there does not apply, I have confirmed that the credential files are created correctly

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 28 (16 by maintainers)

Most upvoted comments

@vrothberg It will not work because this file should be created(copied, generated, linked) in advance, podman doesn’t create a symlink for this file (tested on macos and linux fedora). Temporary solution is disabling buildkit

export DOCKER_BUILDKIT=0

we have a bunch of issues related to incompatibilities with buildkit

also thanks to @vrothberg another solution can be

ln -s ~/.config/containers/auth.json ~/.docker/config.json

I hope it will help somebody

+2
dzirg44 on Jun 21, 2023

podman login --authfile ~/.docker/config.json docker.io

With #20621 (as it is now, subject to change), this will generate a warning, pointing to a new podman login --compat-auth-file ~/.docker/config.json … option.

+1
mtrmac on Nov 8, 2023
Read more comments on GitHub
← podman: Regression: 1.2.0: 'build -t' tags even on error
podman: Remote client on MacOS is not using ssh-agent and is prompting to unlock keys everytime →