moby: Publishing a large port range is very slow (Docker 1.6.0)
Description of problem: When using docker run including the -p publish flag with a large port range, it’s very slow to start a container.
docker version:
Client version: 1.6.0
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 350a636/1.6.0
OS/Arch (client): linux/amd64
Server version: 1.6.0
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 350a636/1.6.0
OS/Arch (server): linux/amd64
docker info:
Containers: 2
Images: 209
Storage Driver: devicemapper
Pool Name: docker-8:6-2623196-pool
Pool Blocksize: 65.54 kB
Backing Filesystem: extfs
Data file:
Metadata file:
Data Space Used: 6.377 GB
Data Space Total: 107.4 GB
Data Space Available: 101 GB
Metadata Space Used: 10.33 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.137 GB
Udev Sync Supported: true
Library Version: 1.02.93 (2015-01-30)
Execution Driver: native-0.2
Kernel Version: 3.19.7-200.fc21.x86_64
Operating System: Fedora 21 (Twenty One)
CPUs: 8
Total Memory: 7.709 GiB
Name: localhost.localdomain
ID: KPCC:PRCE:75J7:BGVS:UOGT:5H2T:55MW:HWP5:EEKN:AIAE:DRCX:TFSE
Username: [redacted]
Registry: [https://index.docker.io/v1/]
uname -a: Linux localhost.localdomain 3.19.7-200.fc21.x86_64 #1 SMP Thu May 7 22:00:21 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Environment details (AWS, VirtualBox, physical, etc.):
Physical on Fedora 21
[root@localhost docker]# docker -v
Docker version 1.6.0, build 350a636/1.6.0
[root@localhost docker]# cat /etc/redhat-release
Fedora release 21 (Twenty One)
[root@localhost docker]# uname -a
Linux localhost.localdomain 3.19.7-200.fc21.x86_64 #1 SMP Thu May 7 22:00:21 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
How reproducible: Every time.
Steps to Reproduce:
- Start a docker container using
docker runwith a-ppublish command with a port range, try 1000 ports e.g.docker run -p 1000-2000:1000-2000 -id centos:centos6 /bin/bash
Actual Results: It takes a full minute to start a container with a thousand ports published for a container that typically takes seconds to start on the same system.
Expected Results: Just a couple seconds to start – preferably almost as fast as publishing just a few ports.
Additional info: Here’s some times for how long it takes to run a container
######## Starting with a single port published.
[root@localhost docker]# time docker run -p 80:80 -id centos:centos6 /bin/bash
01a14d3b2876196a0b209bcb94e7d86d8c32fed40247ea1d463893cc839bd471
real 0m2.740s
user 0m0.030s
sys 0m0.013s
######## Starting with a thousand ports published.
[root@localhost docker]# time docker run -p 1000-2000:1000-2000 -id centos:centos6 /bin/bash
83a672a4487c16be3f5a3ec04b35ad7a5c3a03c303a74d2b72b37fc31e446a78
real 0m56.049s
user 0m0.036s
sys 0m0.016s
######## Or how about 4 ports.
[root@localhost docker]# time docker run -p 4000-4004:4000-4004 -id centos:centos6 /bin/bash
d0253005fc198d47dda5ccb43f6a2bb1cd06ea8066130041bc87622a0e98852e
real 0m2.075s
user 0m0.033s
sys 0m0.012s
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 7
- Comments: 23 (4 by maintainers)
I have same problem with publishing Asterisk container RTP ports. Creating 10k ports in Iptables takes several hours.
Please reopen this issue. Starting a container with a large port range takes forever. For applications like RTP servers, it’s nigh on unusable.
Same problem, coturn default port range (-p 49152-65535:49152-65535/udp) is unusable due to this:
This is still a problem in the newest version . Even with userland-proxy disabled it needs way too long publishing multiple thousands of ports.
I ran the following commands multiple times and I got always about the same result:
Output of
time docker run --rm alpine echo test:Output of
time docker run --rm -p 10000-10100:10000-10100/udp alpine echo test:Output of
time docker run --rm -p 10000-10200:10000-10200/udp alpine echo test:Parts from the output of
iptables --listwhile the container is running:Docker creates one iptables rule per port. The more Ports I forward, the more time consuming it is. Can we maybe change the iptables rules to use port ranges instead?
Output of
docker version:Output of
docker info:Output of
cat /etc/docker/daemon.json:Additional environment details (AWS, VirtualBox, physical, etc.) Physical Server with Ubuntu 16.04
Same issue with Docker version 20.10.6, build 370c289. Trying to open 49152-65535 ports for ejabberd server consumes all memory (10GB) and server hangs-up.
To be clear, there is work being done to overhaul and solve a lot of issues port mapping in docker and we are very aware of the issue around mapping large port ranges.
Same issue with Docker 19.03.8.
Closing this since it is a dup, and is essentially fixed. The issue here is that for each published port Docker has to spin up a userland proxy process to proxy local traffic to the container.
In Docker 1.7 you can disable the userland proxy (which enables hairpin nat), thus significantly speeding up this process. It still takes longer to start/stop a container with lots and lots of published ports, but still much much shorter time than before:
@Netzvamp Depending on your use case, there’s another workaround that could fit your needs: starting the container within host netns (eg.
docker run -d --net=host instrumentisto/coturn). It removes network isolation between your container and the host machine but at least there’s no more iptables & userland-proxy involved, so it’s as fast as starting any other container (~300ms on my computer). However, you might not want to do that for production use cases, and that’s understandable.@indywidualny There’re a bunch of things that need to be refactored in libnetwork, including how iptables rules are managed. Fortunately things are starting to move. So hopefully, this issue will be properly addressed in the not so distant future 🙂
This is also true for recent Docker versions. It’s basically impossible to start 1000 ports.