podman: Podman failed to destroy BTRFS snapshot on container delete
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
bug
Description
ROOTLESS Podman fails to delete BTRFS subvolumes when building an image or deleting a container. This causes a cascade of errors, such as container name re-use errors, as podman believes the container was removed when using podman ps -a however when attempting to re-run the podman run command the user will receive an name re-use error message.
Using SUDO this works just fine.
I would very much assume that this is a configuration issue on my part somewhere, as without privilege elevation using sudo I cannot delete the specified BTRFS by hand using btrfs su delete <path to subvolume> either.
Thanks in advance for your help.
Steps to reproduce the issue:
-
Build a rootless image using the BTRFS driver.
-
Get error message listed below.
Describe the results you received:
ERRO[4128] error deleting build container "de8886e87ab7c7e667426e84a096695f4d434fe8ed42149fb157e7b9a398b906": Failed to destroy btrfs snapshot /home/sbrady/.local/share/containers/storage/btrfs/subvolumes for 561da8272542ab2a71977655a51b5d20c20627fd3a917165d1fe89b0370f4f93: operation not permitted
Error: Failed to destroy btrfs snapshot /home/sbrady/.local/share/containers/storage/btrfs/subvolumes for 561da8272542ab2a71977655a51b5d20c20627fd3a917165d1fe89b0370f4f93: operation not permitted
Describe the results you expected:
BTRFS subvolumes to be deleted on container deletion.
Additional information you deem important (e.g. issue happens only occasionally):
Consistent regardless of the image.
Output of podman version:
podman version 1.5.1
Output of podman info --debug:
debug:
compiler: gc
git commit: ""
go version: go1.12.9
podman version: 1.5.1
host:
BuildahVersion: 1.10.1
Conmon:
package: conmon-2.0.0-2.1.x86_64
path: /usr/bin/conmon
version: 'conmon version 2.0.0, commit: unknown'
Distribution:
distribution: '"opensuse-tumbleweed"'
version: "20190904"
MemFree: 1173884928
MemTotal: 8254943232
OCIRuntime:
package: runc-1.0.0~rc8-1.4.x86_64
path: /usr/bin/runc
version: |-
runc version 1.0.0-rc8
spec: 1.0.1-dev
SwapFree: 2141966336
SwapTotal: 2147483648
arch: amd64
cpus: 4
eventlogger: file
hostname: rocinante
kernel: 5.2.11-1-default
os: linux
rootless: true
uptime: 2h 31m 17.3s (Approximately 0.08 days)
registries:
blocked: null
insecure: null
search:
- docker.io
store:
ConfigFile: /home/sbrady/.config/containers/storage.conf
ContainerStore:
number: 18
GraphDriverName: btrfs
GraphOptions: null
GraphRoot: /home/sbrady/.local/share/containers/storage
GraphStatus:
Build Version: 'Btrfs v5.2.1 '
Library Version: "102"
ImageStore:
number: 16
RunRoot: /var/run/user/1000/containers
VolumePath: /home/sbrady/.local/share/containers/storage/volumes
Package info (e.g. output of rpm -q podman or apt list podman):
podman-1.5.1-1.1.x86_64
Additional environment details (AWS, VirtualBox, physical, etc.): Bare metal install on Intel i7 and spinning rust HDD.
Pastes of Storage.conf and libpod.conf
➜ la ~/.config/containers
total 40K
-rw-r--r-- 1 sbrady users 4.4K Sep 6 14:23 libpod.conf
-rw-r--r-- 1 sbrady users 205 Aug 9 13:30 mounts.conf
drwxr-xr-x 1 sbrady users 14 Aug 9 13:30 oci/
-rw-r--r-- 1 sbrady users 256 Aug 9 13:30 policy.json
-rw-r--r-- 1 sbrady users 1.1K Aug 9 13:30 registries.conf
drwxr-xr-x 1 sbrady users 0 Aug 9 13:30 registries.d/
-rw-r--r-- 1 sbrady users 12K Aug 9 13:30 seccomp.json
-rw-r--r-- 1 sbrady users 5.0K Sep 6 13:21 storage.conf
/home/sbrady/.local/share/containers
├── cache
│ └── blob-info-cache-v1.boltdb
└── storage
├── btrfs
│ └── subvolumes
├── btrfs-containers
│ ├── 1e96b0289a1d5ac651f5c521d325dd2911f73b2a6dffbcbfb2e982310ffbaf05
│ ├── 30e2a4ea384aa36e4a9d5313a89a47efca248f89a6e134f71f0aa952b536b51c
│ ├── 30f3976b8ab272bd229a770b0f0e9807ad8b00798178a6732909da3899308935
│ ├── 3886c1a160034a4c7cae0c59b1a3ee93e0371b837bae4072336b2df16cd2a4cc
│ ├── 3dd0a9edede5dd4fa3a5333665fcb69f45235e5456bb781863f39e41fe0047b7
│ ├── 4f8e7486fbc706acaa2675f3cb8afa32a5e2b742810fea560c740ff7ddadfb86
│ ├── 6e0b92b86a79fb1e8cc6d6c68f7c8e82d9eb3ef96b758c5d3b0a850d5ecdd30a
│ ├── 7051dd05a2a3c0712b92a3d5277b17beedb221c29c7859f7279e778300cc0239
│ ├── 812bfe158ec304e077e6d2e05eb7d3f9f01631e3bc7c6fc49c01d205117bba8f
│ ├── 9e8c3f9cb6b6346f808a89e417d3b365f737cd849ae4e8c41b388f6740640be3
│ ├── b10003959017ff33e909df830fb7acbd87c081bfcf53f997aba6c7afe5040ded
│ ├── c20462222981918a27e00267ba8ba0118c1064b9d2728b529c1ff0c9d75cb238
│ ├── c8bf5970068ed878adadacc10a8e04fd3471f16fa5f95750f501bc9c50cf596b
│ ├── ca09086e22391e95198dd7aa5abc34f168cd64132488e74042c3aaa7860f162c
│ ├── containers.json
│ ├── containers.lock
│ ├── d2878a44fb5ceed65a97f2a3c98f50cc7ec1f4ba02b672d8708278f9c9d1c2b9
│ ├── de8886e87ab7c7e667426e84a096695f4d434fe8ed42149fb157e7b9a398b906
│ ├── e0780ebf9f35bb28407986c191257acdb529d9aa198ca2e5132f06390ac3bf0c
│ ├── e7315b2231cfaefb7991b403ffb18d60d26e25eb1950a090155b3c3be776ea19
│ └── fea6040e4ffbb6fcf85570aee5475fb2a03853a540b8aa89be2d945c6af64290
├── btrfs-images
│ ├── 0868e92e943cba2ce2ed3b5705d9dcd4adeec9da9088ab69b3c44af199072b3c
│ ├── 0ed5811d6d9c68658a20eb354b1917bbf5af162c773eb4edf6168fae00ff09f6
│ ├── 172b73ede26844c52a903bb4e905f18ecbdc1227a6e32b86eb66b60ce999224f
│ ├── 18ffcb379eccf2d03f71066d45bf3d9c6078c8dc2de843eb41361f22aa8e8430
│ ├── 23b52ed766eb03c4151be9e41a0eb2fdce003c910982665b7b92a468bca1c3b3
│ ├── 337ab92e6b8b755823c3363b11948f13441bcb2811988617793642dd1c5c0ac9
│ ├── 5e09dd17175ecdff0b478333a6d5f444ea2314e5f9114a5443d7fc9fec86b834
│ ├── 69c54a0cfa733e6fdb478b5612127feec51153ce066cff4a32f1fdce84bb8af6
│ ├── 6d038d18f5017765cfcbb2262ae3933429e5be0c64f5d70130de8c788791e1d7
│ ├── a8c54eebc7056cd3dfccee64c28c12d7653fbe2b1fe61159f2c08fcfb15110b0
│ ├── b14710b9d573f363bbbad56f0ff69e79b5f229b83daaecaf25d9856a84308df7
│ ├── b151cdb91db489ee8ab7ba84839dd420164692b3031020c8ded00436421715b6
│ ├── cac4ae0c405ea55bed5402512d66abadac6ea01c31e650ffd86af31560ad98bc
│ ├── ced8a8fe165881fdef10a647838751d98e0a3d7aba06316f44bdf28f86e23d25
│ ├── e6da4025fb017e4e79e7339c5953f4c1aa247c11fe7862920d53ef8879341cfa
│ ├── fce289e99eb9bca977dae136fbe2a82b6b7d4c372474c9235adc1741675f587e
│ ├── images.json
│ └── images.lock
├── btrfs-layers
│ ├── 06b85792928655f9c05298c2104b509396a9ae72fe22f739bb004a2ff87f99d0.tar-split.gz
│ ├── 30fa407a2912badb23e133597472bec5d233e439a497d96751f5dcfb8617894e.tar-split.gz
│ ├── 3f194c10a561e3b694e9602b088a4e516eb19ee468992bb5cc6845c717b06b49.tar-split.gz
│ ├── 43316bbb040759a859d32501f20dd865db10b6ee62659fd09aead1a920097982.tar-split.gz
│ ├── 46e841fc16afde233e84ec806cb528e3d0ae0c8ead03f074aa7699f67b8f1b4c.tar-split.gz
│ ├── 51899d997ab4c1f790759deeeaaebb03d3742c34236e5a85499eb302cea6fb7b.tar-split.gz
│ ├── 8297fa3a5e5f1eb097422189d1f7d9046dfcc658008fe395d8b27e93a5954691.tar-split.gz
│ ├── 866d3ed87bbbc6beb540cc52cb79f8efaec0f3a8b7cf554f65c5df8fce524ec0.tar-split.gz
│ ├── 9dae2a8870ddfae25701156bc993448175a4cf5bbc0f537e8563e286a5a37385.tar-split.gz
│ ├── a17440e364015841337f4b80d2a42cf79936cba443ab941525785a7e35f0c0c6.tar-split.gz
│ ├── a2f24cc38b696cc363f1270684da6d5ae40c5b086f6f6978462e7c7f551e4341.tar-split.gz
│ ├── af0b15c8625bb1938f1d7b17081031f649fd14e6b233688eea3c5483994a66a3.tar-split.gz
│ ├── b6b761c5afcb8b69e34a26df1ce16be5d16b3088c838060dcc2528cc3d4cdd5f.tar-split.gz
│ ├── d548e5ae588bc66a2250ed8312145394cb9704ecdf50153fe608b93ec3c15a0f.tar-split.gz
│ ├── layers.json
│ └── layers.lock
├── cache
│ └── blob-info-cache-v1.boltdb
├── libpod
│ └── bolt_state.db
├── mounts
├── storage.lock
├── tmp
└── volumes
├── brave-storage-volume
└── test
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 29 (11 by maintainers)
@t-msn - Would you be willing to submit your change (https://github.com/t-msn/storage/commit/41c2a90841cfc42f1373dac0de240773b405f536) as a Pull Request?
I think the second part needs an issue of its own. I know btrfs and overlayfs can co-exist in production environments with tens of thousands of snapshots. I’m not sure what the nature of this lack of support could be about. There are nuances that can be workload specific where one works better than the other, and even where overlayfs copy-up operation can be made more efficient using cloning (Btrfs has had reflinks since forever, and XFS enables them in the most recent xfsprogs at mkfs time).