podman: Error: database libpod root directory (staticdir) does not match our libpod root directory (staticdir)

Looks pile it is now complaining about volume path, also in containers.conf (I think named volume_path but not sure?). Same procedure for the new error but different key in containers.conf basically.

Perfect! That works for podman at least

For future finders my file looks like

[engine]
static_dir = "/home/lkn/.local/share/containers/storage/libpod"
volume_path = "/home/lkn/.local/share/containers/storage/libpod"

Now trying to access existing containers with distrobox I get an error like

ERRO[0000] Unmounting /home/lkn/.local/share/containers/storage/overlay/f79b472b5b98d8e7a728c7d52415ea60b1043ae60cb1d08ee766f8dc71c08963/merged: invalid argument 
Error: unable to start container "ff34fb71718140a638f7c834ed6d116f413d8a50f8bfbc88307a38d717f7ff52": mounting storage for container ff34fb71718140a638f7c834ed6d116f413d8a50f8bfbc88307a38d717f7ff52: creating overlay mount to /home/lkn/.local/share/containers/storage/overlay/f79b472b5b98d8e7a728c7d52415ea60b1043ae60cb1d08ee766f8dc71c08963/merged, mount_data="lowerdir=/home/lkn/.local/share/containers/storage/overlay/l/6I7L4CUQJIQPFUYHFX5OZBL4JK:/home/lkn/.local/share/containers/storage/overlay/l/6I7L4CUQJIQPFUYHFX5OZBL4JK/../diff1:/home/lkn/.local/share/containers/storage/overlay/l/7JTTU5TJDPKSG76SE4CVJCD5OT:/home/lkn/.local/share/containers/storage/overlay/l/BFAWGXFYIDBJFEOEEW66NT5TUI,upperdir=/home/lkn/.local/share/containers/storage/overlay/f79b472b5b98d8e7a728c7d52415ea60b1043ae60cb1d08ee766f8dc71c08963/diff,workdir=/home/lkn/.local/share/containers/storage/overlay/f79b472b5b98d8e7a728c7d52415ea60b1043ae60cb1d08ee766f8dc71c08963/work,userxattr,context=\"system_u:object_r:container_file_t:s0:c1022,c1023\"": invalid argument

But I’m happy to take this elsewhere if it’s not directly related to podman 😄

EDIT: Managed to rescue, turns out it was SELinux. Turning it off but not fully disabled allowed me to carry on using my old containers; thanks @mheon you’re a life-saver 😄

There are no consequences even after the fix lands. The two paths are symlinked together, we’re just forcing another version be used when doing database checks.

Thank you very much! I was having the same problem and this issue helped me solve it.

In my environment, there was no directory, not even containers.conf, but if I create the directory and file, it should be OK. The containers.conf mentioned in this comment is in the following path, /home/lkn/.config/containers/containers.conf

I rebased from Fedora 39 Kinoite to Rawhide Kinoite to do some testing and this affects my ability to run a toolbox:

toolbox
Error: failed to get the Podman version
...
podman --version
podman version 4.8.1

I couldn’t reproduce this on my Fedora 39 Silverblue:

rishi@fedora:~$ podman version
Client:       Podman Engine
Version:      4.8.0
API Version:  4.8.0
Go Version:   go1.21.4
Built:        Tue Nov 28 10:58:32 2023
OS/Arch:      linux/amd64
rishi@fedora:~$ 
rishi@fedora:~$ echo $HOME
/var/home/rishi
rishi@fedora:~$ 

… but clearly a lot of people can. Based on these two comments, I think this affects those who have been upgrading their installations in-place for a while.

I usually reinstall once every year, and I remember that we changed the default value of HOME from /home/foo to /var/home/foo at some point to avoid relying on the symbolic link.

Setting static_dir in containers.conf is more likely to work. If your error message is:

Error: database libpod root directory (staticdir) "/home/<user>/.local/share/containers/storage/libpod" does not match our libpod root directory (staticdir) "/var/home/<user>/.local/share/containers/storage/libpod": database configuration mismatch

Then set static_dir to the “database libpod root directory” (in this case /home/<user>/.local/share/containers/storage/libpod)