moby: race conditions lead to duplicate docker networks
When using docker network create
, or via docker-compose
- concurrent calls will create a duplicate network.
Output of docker version
:
Client:
Version: 1.10.2
API version: 1.22
Go version: go1.5.3
Git commit: c3959b1
Built: Mon Feb 22 21:37:01 2016
OS/Arch: linux/amd64
Server:
Version: 1.10.2
API version: 1.22
Go version: go1.5.3
Git commit: c3959b1
Built: Mon Feb 22 21:37:01 2016
OS/Arch: linux/amd64
vagrant@vagrant-ubuntu-tr
Output of docker info
:
Containers: 5
Running: 5
Paused: 0
Stopped: 0
Images: 24
Server Version: 1.10.2
Storage Driver: devicemapper
Pool Name: docker-8:1-262711-pool
Pool Blocksize: 65.54 kB
Base Device Size: 10.74 GB
Backing Filesystem: ext4
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 2.007 GB
Data Space Total: 107.4 GB
Data Space Available: 37.2 GB
Metadata Space Used: 4.063 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.143 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/231072.231072/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
Metadata loop file: /var/lib/docker/231072.231072/devicemapper/devicemapper/metadata
Library Version: 1.02.77 (2012-10-15)
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
Volume: local
Network: bridge null host
Kernel Version: 3.13.0-67-generic
Operating System: Ubuntu 14.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.955 GiB
Name: vagrant-ubuntu-trusty-64
ID: XHSN:N4BF:J6HX:PVNW:GP7J:QES5:QYLN:W7YY:NFJK:4OKF:M2GH:6GHD
WARNING: No swap limit support
Provide additional environment details (AWS, VirtualBox, physical, etc.): vagrant box ubuntu trusty, running on virtual-box 4.3.34 on mac mini OSX 10.10.5
List the steps to reproduce the issue:
docker network create xyz & docker network create xyz
Describe the results you received:
network xyz
is created twice:
vagrant@vagrant-ubuntu-trusty-64:~$ docker network ls
NETWORK ID NAME DRIVER
f0be1507dc2b xyz bridge
f091a7d18a7f xyz bridge
9bfb77c3d0c1 none null
fb8f222d4bb7 host host
Describe the results you expected: Expected is a single network. This causes confusion with connected services.
Provide additional info you think is important: The original issue was caused by 3 docker-compose calls that started concurrently, each launching different service in a compose file.; this causes the network to be created 3 times, and each service was on a different instance.
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Reactions: 6
- Comments: 19 (9 by maintainers)
Commits related to this issue
- Resolve #20648 using a coarse grained lock on check duplicate network name and create new network Signed-off-by: Ron Peleg <ron.peleg@gmail.com> — committed to rpeleg1970/docker by rpeleg1970 8 years ago
- resolve #20648 using a fine grained lock on natwork name Signed-off-by: Ron Peleg <ron.peleg@gmail.com> — committed to rpeleg1970/docker by rpeleg1970 8 years ago
- using pkg/locaker for fine grained locking on name, fixes #20648 Signed-off-by: Ron Peleg <ron.peleg@gmail.com> Signed-off-by: ronp@winter <ron.peleg@trusteer.com> — committed to rpeleg1970/docker by rpeleg1970 8 years ago
- fixed #20648 using named based locking from pkg/locker Signed-off-by: Ron Peleg <ron.peleg@gmail.com> — committed to rpeleg1970/docker by rpeleg1970 8 years ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
- libnet: Make sure network names are unique Fixes #18864, #20648, #33561, #40901. [This GH comment][1] makes clear network name uniqueness has never been enforced due to the eventually consistent nat... — committed to akerouanton/docker by akerouanton 10 months ago
#46251 has been merged. Starting with the next release, it won’t be possible to create networks with duplicate names (even when doing concurrent API calls). So let me close this one.
This is pretty frustrating to deal with for situations / tools like compose (which seems to have not really solved this?)
The compose team seems to be repeatedly stating that this is a bug in the engine that should be solved here, at most they seem to be passing checkDuplicate, which appears to be best effort only.
https://github.com/moby/moby/issues/40901#issue-612173611 suggests that if two networks with the same name are created, even if you use the IDs you will not be able to start containers attached to them …
Is this really still intended / supported behavior? It seems pretty surprising / inconsistent with volumes and containers, and a bit user-hostile.
@vdemeester This is by design https://github.com/moby/moby/issues/18864#issuecomment-167006094
If you create the network in a script, you can easily solve the duplicated network issue executing a
run-one
command to create the network (instead of calling it directly), to make sure that there aren’t 2 processes trying to create the same network.More details can be seen in the SO answer: https://stackoverflow.com/a/68448059/4850646
This may not be so easily achievable with
docker-compose
tough, unless you run it from a script that you can easily change and you know the names of the networks beforehand.That said, this is just a workaround, and I think that a proper solution should be in the docker engine (if having duplicated named networks is by design, I would expect at least some flag to disable this behaviour, or some CLI & compose option to make sure that the network name is unique, even if by default duplicates are allowed).