compose: docker-compose V2 fails for port range and replicas>1 where V1 would succeed
Description
Using multiple replicas of a container where each replica should use one of the ports selected by a port-range:
version: "3.4"
services:
dummy:
image: busybox
command: ["sh", "-c", "while true ; do sleep 3 ; date ; done"]
deploy:
replicas: 2
ports:
- "6005-6015:5000"
When starting up these containers with docker-compose V1 the container instances use port-mapping where each container picks one host-port from the selected port range to map to internal container port.
$ docker-compose ps
Name Command State Ports
----------------------------------------------------------------------------------------------------------------
docker-compose-issue_dummy_1 sh -c while true ; do slee ... Up 0.0.0.0:6007->5000/tcp,:::6007->5000/tcp
docker-compose-issue_dummy_2 sh -c while true ; do slee ... Up 0.0.0.0:6008->5000/tcp,:::6008->5000/tcp
When starting up these containers with docker-compose V2 the containers use all ports from the selected port range to map to internal container port, resulting in error like the following for the second instance of that service:
Error response from daemon: driver failed programming external connectivity on endpoint docker-compose-issue_dummy_1 (6d9cff4937b3b6946e0bb949a36362cf91477f841be34ac9bd2693453b6e8837): Bind for 0.0.0.0:6009 failed: port is already allocated
Steps to reproduce the issue:
- Switch to docker-compose V1,
docker-compose disable-v2
- Startup containers,
docker-compose up -d
- List running containers and the port mapping they use,
docker-compose ps
- Shutdown service,
docker-compose down
- Switch to docker-compose V2,
docker-compose enable-v2
- Startup containers,
docker-compose up -d
- docker-compose fails to startup because of port mapping clash
Describe the results you received:
docker-compose.yaml only seems to work for V1 and not V2.
Describe the results you expected:
I would expect that docker-compose.yaml can be run by V1 and V2 alike.
Output of docker version
:
Client:
Cloud integration: 1.0.17
Version: 20.10.7
API version: 1.41
Go version: go1.16.4
Git commit: f0df350
Built: Wed Jun 2 11:56:22 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: b0f5bc3
Built: Wed Jun 2 11:54:58 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.6
GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc:
Version: 1.0.0-rc95
GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info
:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
compose: Docker Compose (Docker Inc., 2.0.0-beta.3)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 30
Running: 10
Paused: 0
Stopped: 20
Images: 136
Server Version: 20.10.7
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.25-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 3
Total Memory: 11.7GiB
Name: docker-desktop
ID: 65BF:GFI6:S3ZY:URLI:EHO5:GFN5:PT7Y:Q7JW:ACRV:FJCE:YM5X:2YFM
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 159
Goroutines: 157
System Time: 2021-06-24T14:52:06.2642643Z
EventsListeners: 5
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS ECS, Azure ACI, local, etc.):
local, MacOS BigSur 11.4
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 23
- Comments: 26
I’m not sure if this is the same problem, but researching how to solve my own problem took me here. I was having a similar problem on a Mac,
and I was able to fix it by checking:
and then turning off the
Airplay Receiver
Service inSystem Preferences -> Sharing
If I am not mistaken, the “bug” is still here.
It’s still happening. (m1 mac, macOS 12.1 monterey)
I’d like to fix this, but don’t know where to begin. Any advice?
Docker version 20.10.13, build a224086 Docker Compose version v2.3.3
I’m seeing this issue also.
@hacdias @Hazmi35 If you run the
docker compose up
several times (one for each replica) you can eventually get the system running with the replicas and port mappings.Currently having to remove the replica config an repeat the container instance config for each replica I need.
This issue needs looking at…its been open for a very long time with a significant bug in the use of docker-compose when testing a scaling setup. I’ve moved to Kubernetes/skaffold.
It seems to be more of a timing issue afaik. I have seen the exact same docker-compose work if there is a delay (in my case due to high cpu load) so that the first container starts and it takes a second or two before the second container starts. When there is almost no delay (just some milliseconds) I get the bind-error.
Hi, I keep facing this issue with docker compose 2.15.1 with following setting
running
docker compose up
starts the first instance but shows the error message (Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:8081 -> 0.0.0.0:0: listen tcp 0.0.0.0:8081: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
), running it a second time make my 2 replicas working as expected (confirmed withdocker ps
)