moby: Swarm service, with overlay network, fails to remove all containers

Output of docker version:

Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:02:53 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:02:53 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 1.12.1
Storage Driver: overlay2
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null bridge host overlay
Swarm: active
 NodeID: 0phf7j5medrlnam24hcki16db
 Is Manager: true
 ClusterID: 0ol98k8pblzgb1xpqeuuqn27f
 Managers: 1
 Nodes: 3
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: 10.20.21.1
Runtimes: runc
Default Runtime: runc
Security Options:
Kernel Version: 4.6.0-0.bpo.1-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.911 GiB
Name: mc-docker1
ID: RG6N:UCJN:AT5K:AYYN:ZPH6:6IAN:ARJZ:TWAD:ZFTZ:BSYC:A4AN:TTP3
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
Insecure Registries:
 127.0.0.0/8

Steps to reproduce the issue:

1. docker network create -d overlay sleeper_nw
2. docker service create --name sleeper --replicas 20 --network sleeper_nw urlund/sleeper
3. docker service rm sleeper

Describe the results you received: Output from docker ps -a:

Node 1:

CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS                       PORTS               NAMES
9b79d94bcebb        urlund/sleeper:latest   "/run.sh"           41 seconds ago      Exited (137) 7 seconds ago                       sleeper.20.5shcg82g0p2sn8hkqidx6yuim
239a00862bc4        urlund/sleeper:latest   "/run.sh"           41 seconds ago      Exited (137) 7 seconds ago                       sleeper.2.7vm1a1u41fwpf3mjrviphsfyo
274ccd13294e        urlund/sleeper:latest   "/run.sh"           41 seconds ago      Exited (137) 7 seconds ago                       sleeper.18.1qzllrci7j6rzpjdvk01fxip7
7837971e8e4d        urlund/sleeper:latest   "/run.sh"           41 seconds ago      Exited (137) 7 seconds ago                       sleeper.6.dzrkydszgnvd4mzyy37fyxarc

Node 2:

CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS                        PORTS               NAMES
854ef7a322b7        urlund/sleeper:latest   "/run.sh"           44 seconds ago      Exited (137) 10 seconds ago                       sleeper.19.609h64iauf7oa1i95o0qi7qqk
fcfcf0208ebf        urlund/sleeper:latest   "/run.sh"           44 seconds ago      Exited (137) 10 seconds ago                       sleeper.15.dpxzl9lysf6koofzhld6o8hnq

Node 3:

CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS                        PORTS               NAMES
32518f0c9ce2        urlund/sleeper:latest   "/run.sh"           45 seconds ago      Exited (137) 12 seconds ago                       sleeper.9.1k6snzf3xnjjfayoq3fhfvdtu
bf15e1d18a8a        urlund/sleeper:latest   "/run.sh"           45 seconds ago      Exited (137) 12 seconds ago                       sleeper.5.bqf8dn47yqcosd45uk0skzvyj
fc406aef2f9c        urlund/sleeper:latest   "/run.sh"           45 seconds ago      Exited (137) 12 seconds ago                       sleeper.12.68hlgm025wtur6xa57v8gfi8i
d5a9359d120e        urlund/sleeper:latest   "/run.sh"           45 seconds ago      Exited (137) 12 seconds ago                       sleeper.7.0ql6whgya5753ckk12nahqx5u
46035a959fbe        urlund/sleeper:latest   "/run.sh"           45 seconds ago      Exited (137) 12 seconds ago                       sleeper.10.e5z07uzu46z9c051iusx6qpqf

Describe the results you expected: I would expect all containers to be removed.

Additional information you deem important (e.g. issue happens only occasionally): You should not be able to reproduce this without creating/configuring an overlay network.