moby: Cannot destroy container: Driver zfs failed to remove root filesystem

Reproduce by

# docker rm -f cont_name_1

Error response from daemon: Cannot destroy container cont_name_1: Driver zfs failed to remove root filesystem 43d...: exit status 1: "/sbin/zfs zfs destroy -r rpool/ROOT/pve-1/43d..." => cannot destroy 'rpool/ROOT/pve-1/43d...': dataset is busy

# docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                      PORTS                                                                 NAMES
970f876a5d8f        tianon/true                  "/true"                  2 days ago          Exited (0) 2 days ago                                                                             DB-DATA_1
1e6f1c42aa82        mariadb                      "/docker-entrypoint.s"   7 days ago          Dead

I get this error very frequently when deleting stopped containers on ZFS. Usually the containers show up in docker ps -a as STATUS ‘Dead’ without a name. Deleting them again by container ID will produce the same error. The containers will only disappear after restarting the docker service.

The system is running Debian 7 with a Proxmox 3.4 kernel Linux 3.10.0-11-pve x86_64 (this is the most recent kernel available for the stable version of Proxmox). The filesystem is ZFS Raid1

 Docker Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:28:37 UTC 2015
 OS/Arch:      linux/amd64


Containers: 28
Images: 378
Storage Driver: zfs
 Zpool: rpool
 Zpool Health: ONLINE
 Parent Dataset: rpool/ROOT/pve-1
 Space Used By Parent: 77297631232
 Space Available: 33526665216
 Parent Quota: no
 Compression: lz4
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-11-pve
Operating System: Debian GNU/Linux 7 (wheezy)
CPUs: 8
Total Memory: 15.66 GiB
Name: prox
ID: YCEI:Q4MY:ZPMZ:A7IM:D6TG:UI65:2ZVK:MVBX:GZUY:TSL7:5AD5:SFTY
Registry: https://index.docker.io/v1/

About this issue

Original URL
State: closed
Created 9 years ago
Comments: 27 (13 by maintainers)

Most upvoted comments

ubuntu 20.0.4 encountered the same problem, kernel 5.4.0

moorema on May 8, 2020

@xZero707 different error; dataset does not exist (this ticket is discussing dataset is busy errors. I see that error being mentioned in https://github.com/moby/moby/issues/30993 (which may give some instructions that could help)

thaJeztah on May 18, 2020

Same problem on Ubuntu 21.04, Linux jo 5.11.0-16-generic

$ docker rm 67
Error response from daemon: container 677e3db0958ddbd5e3eb862995189bc79db4b37e1e5b57b142817397b8779835: driver "zfs" failed to remove root filesystem: exit status 1: "/usr/sbin/zfs fs destroy -r rpool/ROOT/ubuntu_mqaenc/var/lib/0ac372d9113a3a96eb3700dc1fccf49766b2737fdfc9f3a9f7ba8fb3aa4ade42" => cannot open 'rpool/ROOT/ubuntu_mqaenc/var/lib/0ac372d9113a3a96eb3700dc1fccf49766b2737fdfc9f3a9f7ba8fb3aa4ade42': dataset does not exist

$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="21.04 (Hirsute Hippo)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 21.04"
VERSION_ID="21.04"

$ docker --version   
Docker version 20.10.6, build 370c289

Joeyscat on Apr 29, 2021

Same problem here using Ubuntu 20.4 kernel 5.4.0-65-generic. To work around the problem, first, unmount the zfs filesystem and then you will be able to remove the container. sudo zfs unmount /tank/myzfs docker rm -f myContainer

elvyscruz on Feb 10, 2021

Thanks! Let me go ahead and close

thaJeztah on Jan 8, 2018