podman: IPAM error: failed to open database /run/user/1000/containers/networks/ipam.db
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
Starting a rootless container returns the following error:
Error: unable to start container 57fa8deff8938fe7e39843f1cacd5211e6ff0e1c6c3e1a83c2fd914a72e14526: IPAM error: failed to open database /run/user/1000/containers/networks/ipam.db: open /run/user/1000/containers/networks/ipam.db: no such file or directory
exit code: 125
The file indeed does not exist:
$ ls -lh /run/user/1000/containers/networks/ipam.db
ls: cannot access '/run/user/1000/containers/networks/ipam.db': No such file or directory
The issue persists across completely nuking podman with podman system reset --force.
Additional information you deem important (e.g. issue happens only occasionally):
Output of podman version:
Client: Podman Engine
Version: 4.1.0
API Version: 4.1.0
Go Version: go1.17.5
Git Commit: e4b03902052294d4f342a185bb54702ed5bed8b1
Built: Wed Jun 15 18:04:43 2022
OS/Arch: linux/amd64
Output of podman info --debug:
host:
arch: amd64
buildahVersion: 1.26.1
cgroupControllers:
- cpuset
- cpu
- io
- memory
- hugetlb
- pids
- misc
cgroupManager: cgroupfs
cgroupVersion: v2
conmon:
package: app-containers/conmon-2.0.30
path: /usr/libexec/podman/conmon
version: 'conmon version 2.0.30, commit: v2.0.30'
cpuUtilization:
idlePercent: 66.64
systemPercent: 6.92
userPercent: 26.44
cpus: 4
distribution:
distribution: gentoo
version: unknown
eventLogger: file
hostname: RPC
idMappings:
gidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 1065536
size: 65536
uidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 1065536
size: 65536
kernel: 5.16.9RMOD-ga9524784f43d
linkmode: dynamic
logDriver: k8s-file
memFree: 617893888
memTotal: 16647462912
networkBackend: netavark
ociRuntime:
name: crun
package: app-containers/crun-1.4.4
path: /usr/bin/crun
version: |-
crun version 1.4.4
commit: 6521fcc5806f20f6187eb933f9f45130c86da230
spec: 1.0.0
+SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
os: linux
remoteSocket:
path: /var/run/user/1000/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: false
slirp4netns:
executable: /usr/bin/slirp4netns
package: app-containers/slirp4netns-1.2.0
version: |-
slirp4netns version 1.2.0
commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
libslirp: 4.3.1
SLIRP_CONFIG_VERSION_MAX: 3
libseccomp: 2.5.1
swapFree: 1072164864
swapTotal: 1073737728
uptime: 108h 45m 40.24s (Approximately 4.50 days)
plugins:
log:
- k8s-file
- none
- passthrough
network:
- bridge
- macvlan
volume:
- local
registries:
docker.io:
Blocked: false
Insecure: false
Location: docker.io
MirrorByDigestOnly: false
Mirrors: null
Prefix: docker.io
PullFromMirror: ""
search:
- docker.io
store:
configFile: /home/rafael/.config/containers/storage.conf
containerStore:
number: 2
paused: 0
running: 0
stopped: 2
graphDriverName: vfs
graphOptions: {}
graphRoot: /home/rafael/.local/share/containers/storage
graphRootAllocated: 983709065216
graphRootUsed: 739086147584
graphStatus: {}
imageCopyTmpDir: /var/tmp
imageStore:
number: 2
runRoot: /run/user/1000/containers
volumePath: /home/rafael/.local/share/containers/storage/volumes
version:
APIVersion: 4.1.0
Built: 1655309083
BuiltTime: Wed Jun 15 18:04:43 2022
GitCommit: e4b03902052294d4f342a185bb54702ed5bed8b1
GoVersion: go1.17.5
Os: linux
OsArch: linux/amd64
Version: 4.1.0
Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)
Yes
Additional environment details (AWS, VirtualBox, physical, etc.):
Gentoo Linux running kernel 5.16.9.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 27 (14 by maintainers)
Commits related to this issue
- rootless netns: eval symlink for XDG_RUNTIME_DIR When we bind mount the old XDG_RUNTIME_DIR to the new fake /run it will cause issues when the XDG_RUNTIME_DIR is a symlink since they do not exists in... — committed to Luap99/libpod by Luap99 2 years ago
- rootless netns: eval symlink for XDG_RUNTIME_DIR When we bind mount the old XDG_RUNTIME_DIR to the new fake /run it will cause issues when the XDG_RUNTIME_DIR is a symlink since they do not exists in... — committed to Luap99/libpod by Luap99 2 years ago
- rootless netns: eval symlink for XDG_RUNTIME_DIR When we bind mount the old XDG_RUNTIME_DIR to the new fake /run it will cause issues when the XDG_RUNTIME_DIR is a symlink since they do not exists in... — committed to Luap99/libpod by Luap99 2 years ago
- rootless netns: eval symlink for XDG_RUNTIME_DIR When we bind mount the old XDG_RUNTIME_DIR to the new fake /run it will cause issues when the XDG_RUNTIME_DIR is a symlink since they do not exists in... — committed to Luap99/libpod by Luap99 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
- Eval symlinks on XDG_RUNTIME_DIR Partial Fix for https://github.com/containers/podman/issues/14606 [NO NEW TESTS NEEDED] Signed-off-by: Daniel J Walsh <dwalsh@redhat.com> — committed to rhatdan/podman by rhatdan 2 years ago
Fixed in https://github.com/containers/podman/pull/15918
We can fix this, a potential easy fix would be to call EvalSymlink on the XDG_RUNTIME_DIR before using it.
Likely the problem is the use of /var/run… instead of just /run and not the mount propagation.