moby: Container removal broken with ZFS as of 17.11.0-ce
Description
As of Docker 17.11.0-ce container removal is broken when used with the ZFS storage driver.
Steps to reproduce the issue:
docker run -it --name bug1 --rm debian bash
in one terminal windowdocker run -it --name bug2 --rm debian bash
in another terminal windowdocker stop bug1
in a third terminal windowdocker ps
and notice Status = Removal In Progressdocker rm bug1
- grep in /proc/*/mounts
- Notice that docker-containerd-shim (of the other container) holds the mount.
Describe the results you received:
[timwolla@~]docker rm bug1
Error response from daemon: driver "zfs" failed to remove root filesystem for 0ba97ff3ff17317fce4bbe3ae6cc63fab0247a9689b4ee26f1ff6d2c0d4408bc: exit status 1: "/sbin/zfs zfs destroy -r scratch/docker/7a0e73a9714f4a9ebf8bda56ca01c026a7ce29c1275900403ce99b55038e1834" => cannot destroy 'scratch/docker/7a0e73a9714f4a9ebf8bda56ca01c026a7ce29c1275900403ce99b55038e1834': dataset is busy
[timwolla@~ [1]]grep 7a0e73a9714f4a9ebf8bda56ca01c026a7ce29c1275900403ce99b55038e1834 /proc/*/mounts
/proc/8625/mounts:scratch/docker/7a0e73a9714f4a9ebf8bda56ca01c026a7ce29c1275900403ce99b55038e1834 /scratch/docker/zfs/graph/7a0e73a9714f4a9ebf8bda56ca01c026a7ce29c1275900403ce99b55038e1834 zfs rw,relatime,xattr,noacl 0 0
[timwolla@~]ps aux |grep 8625
root 8625 0.0 0.1 438744 13208 ? Sl 19:26 0:00 docker-containerd-shim --namespace moby --workdir /scratch/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/e62dc25804ced0d0a222d3eca67210201bd8ddf3c17756a03b9bbedc796a2cc7 --address /var/run/docker/containerd/docker-containerd.sock --runtime-root /var/run/docker/runtime-runc
timwolla 8871 0.0 0.0 15776 984 pts/22 S+ 19:27 0:00 grep --color=auto 8625
Describe the results you expected:
I expected a successful removal of the container.
Additional information you deem important (e.g. issue happens only occasionally):
Can be reliably reproduced as of Docker 17.11. My unit file is modified as follows:
[Unit]
Requires=zfs.target
After=zfs.target
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --icc=true -g /scratch/docker/ -H fd:// --storage-driver=zfs --live-restore --init
Output of docker version
:
Client:
Version: 17.11.0-ce
API version: 1.34
Go version: go1.8.3
Git commit: 1caf76c
Built: Mon Nov 20 18:37:39 2017
OS/Arch: linux/amd64
Server:
Version: 17.11.0-ce
API version: 1.34 (minimum version 1.12)
Go version: go1.8.3
Git commit: 1caf76c
Built: Mon Nov 20 18:36:09 2017
OS/Arch: linux/amd64
Experimental: false
Output of docker info
:
Containers: 2
Running: 1
Paused: 0
Stopped: 1
Images: 173
Server Version: 17.11.0-ce
Storage Driver: zfs
Zpool: scratch
Zpool Health: ONLINE
Parent Dataset: scratch/docker
Space Used By Parent: 9999368192
Space Available: 16792281088
Parent Quota: no
Compression: on
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 992280e8e265f491f7a624ab82f3e238be086e49
runc version: 0351df1c5a66838d0c392b4ac4cf9450de844e2d
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-101-generic
Operating System: Ubuntu 16.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.773GiB
Name: *snip*
ID: *snip*
Docker Root Dir: /scratch/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: true
WARNING: No swap limit support
Additional environment details (AWS, VirtualBox, physical, etc.):
My developer Laptop, running Ubuntu 16.04 LTS.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 5
- Comments: 18 (18 by maintainers)
https://github.com/moby/moby/pull/35674 was merged; I’ll go ahead and close this one