moby: Setting up iptables rules with nftables is a multitude slower than with iptables

Moving this to a separate issue

Running inside the development container from the “bump go 1.13” branch; https://github.com/moby/moby/pull/39549

More details can be found in my comments on https://github.com/moby/moby/pull/39549#issuecomment-513468382

make binary shell
make install

Switch to iptables-legacy:

update-alternatives --set iptables /usr/sbin/iptables-legacy   || true
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy || true
update-alternatives --set arptables /usr/sbin/arptables-legacy || true
update-alternatives --set ebtables /usr/sbin/ebtables-legacy   || true

On a DigitalOcean droplet (4CPU, 8GB, Ubuntu 16.04, 4.4.0-154-generic);

dockerd
INFO[2019-07-22T15:29:22.489882236Z] Starting up            
INFO[2019-07-22T15:29:22.748612702Z] API listen on /var/run/docker.sock

time docker network create foo
6b1cf1f980f2956453bb130b57181b9b9425e7084478e0c464281e2594c7e3c2

real	0m0.066s
user	0m0.020s
sys	0m0.000s

On Docker for Mac (4CPU, 2GB, 4.9.125-linuxkit):

INFO[2019-07-22T15:37:29.043474200Z] Starting up     
INFO[2019-07-22T15:37:29.343360800Z] API listen on /var/run/docker.sock

time docker network create foo
bd17e7754e5f506a14a2b1a2bea7b1d738af34cc1ff7cc92a12a5361055fd32b

real	0m0.052s
user	0m0.000s
sys	0m0.000s

Switch to iptables-nft:

update-alternatives --set iptables /usr/sbin/iptables-nft   || true
update-alternatives --set ip6tables /usr/sbin/ip6tables-nft || true
update-alternatives --set arptables /usr/sbin/arptables-nft || true
update-alternatives --set ebtables /usr/sbin/ebtables-nft   || true

On a DigitalOcean droplet (4CPU, 8GB, Ubuntu 16.04, 4.4.0-154-generic);

INFO[2019-07-22T15:29:56.853611304Z] Starting up                                  
INFO[2019-07-22T15:29:58.697647232Z] API listen on /var/run/docker.sock

time docker network create bar
4f6808b29aba81f89469934440d6515a12987fcc4a6aecc6787907bb4361dc8f

real	0m0.567s
user	0m0.016s
sys	0m0.028s

On Docker for Mac (4CPU, 2GB, 4.9.125-linuxkit):

INFO[2019-07-22T15:38:38.894409400Z] Starting up 
INFO[2019-07-22T15:38:47.071252100Z] API listen on /var/run/docker.sock

time docker network create bar
1f677f98b9de2818420e4ad6738302b264efa074f1f1805b4ea14cfcc45cd57e

real	0m1.609s
user	0m0.000s
sys	0m0.000s

About this issue

Original URL
State: open
Created 5 years ago
Comments: 19 (15 by maintainers)

Links to this issue

No networking for stacks - Swarm - Docker Community Forums

Most upvoted comments

The slowdown is at least a factor of 20

root@86a69485a1d7:/# update-alternatives --set iptables /usr/sbin/iptables-legacy
root@86a69485a1d7:/# time for i in range{1..100}; do iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -D FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;done

real	0m0.620s
user	0m0.000s
sys	0m0.064s
root@86a69485a1d7:/# time for i in range{1..500}; do iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -D FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;done

real	0m3.695s
user	0m0.000s
sys	0m0.300s
root@86a69485a1d7:/# update-alternatives --set iptables /usr/sbin/iptables-nft   
update-alternatives: using /usr/sbin/iptables-nft to provide /usr/sbin/iptables (iptables) in manual mode
root@86a69485a1d7:/# time for i in range{1..100}; do iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -D FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;done

real	0m13.281s
user	0m0.396s
sys	0m0.264s
root@86a69485a1d7:/# time for i in range{1..500}; do iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -D FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;iptables -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT;done

real	1m12.672s
user	0m4.032s
sys	0m4.300s

arkodg on Jul 22, 2019

detecting the default backend (nft or iptables-legacy) on host and pushing the appropriate rule via nft or iptables in Docker might be the best path to take IMHO

arkodg on Jul 22, 2019

Hi @unclejack, this issue is now hitting end users using Docker Desktop for Mac. Do you have any update on your investigation?

stanleymho on Jun 25, 2021

I’m investigating loading rules in bulk and generating rules for the right firewall implementation.

unclejack on Jul 24, 2019