moby: Docker container cannot connect to docker host on it's own exposed port if INPUT chain policy is DROP

Description

I’ve got a docker container running in it’s own bridge network on a docker host <dockerhost>. The containers port 8080 is exposed. I’ve got iptables default policy on the host set to DROP. Docker daemon manages the docker iptables rules.

If I run curl outside my container to http://<dockerhost>:8080 I get HTTP status 200 and the correct website my container is running.

However the same curl from inside my container on that host (to both the IP or hostname) fails to connect:

curl: (7) Failed to connect to <dockerhost> on port 8080: Host is unreachable This doesn’t happen when trying this from a Docker container from another host or from the host itself.

I can solve this by adding the following iptables rule: sudo iptables -I INPUT 1 -i <docker-bridge-interface> -j ACCEPT Where <docker-bridge-interface> is the name of the bridge interface, in which the docker container is running.

Now I can do curl -v dockerhost:port from my container running in the docker-bridge-interface network and exposing above port on the dockerhost.

I think this should be solved by docker: whenever a bridge network is created, a firewall rule should be set to allow the containers in this network to reach the host on it’s own exposed (host) port. Reason we need this is that some webapplications use a baseurl instead relative urls internally.

Steps to reproduce the issue:

$ sudo iptables -P INPUT DROP
$ docker network create -d bridge mynetwork
$ docker run -d -p 8000:8000 --net=mynetwork --name=revealjs amouat/revealjs:latest 
$ docker exec -ti revealjs /bin/bash
# curl -v <dockerhost-ip>:8000
curl: (7) Failed to connect to <dockerhost-ip> port 8000: Connection refused
Or depending on your overall firewall rules, you could also get a timeout.

Describe the results you received:

curl -v http://curie1.ccveu.local:8080
* Rebuilt URL to: http://curie1.ccveu.local:8080/
*   Trying 192.168.56.5...
* connect to 192.168.56.5 port 8080 failed: Connection refused
* Failed to connect to curie1.ccveu.local port 8080: Connection refused
* Closing connection 0
curl: (7) Failed to connect to curie1.ccveu.local port 8080: Connection refused

This doesn’t happen when trying this from a Docker container from another host or from the host itself.

Describe the results you expected:

curl -v http://curie1.ccveu.local:8080
* Rebuilt URL to: http://curie1.ccveu.local:8080/
*   Trying 192.168.56.5...
* Connected to curie1.ccveu.local (192.168.56.5) port 8080 (#0)
> GET / HTTP/1.1
> Host: curie1.ccveu.local:8080
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< Server: Apache-Coyote/1.1
< X-AREQUESTID: 477x3x1
< Location: /startup.jsp?returnTo=%2Fdefault.jsp
< Content-Type: text/html;charset=UTF-8
< Content-Length: 0
< Date: Thu, 27 Oct 2016 07:57:49 GMT
< 
* Connection #0 to host curie1.ccveu.local left intact

Additional information you deem important (e.g. issue happens only occasionally):

I can solve this by adding an iptables rule to allow connections from docker network to host: $ sudo iptables -I INPUT 1 <interfacename-mynetwork> -j ACCEPT

However, IMHO docker should add this rule when a bridge or other network is created.

Output of docker version:

Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 8
 Running: 6
 Paused: 0
 Stopped: 2
Images: 15
Server Version: 1.12.1
Storage Driver: aufs
 Root Dir: /data/docker/docker/aufs
 Backing Filesystem: extfs
 Dirs: 187
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null bridge host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.0-28-generic
Operating System: Ubuntu 16.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.954 GiB
Name: curie.ccveu.local
ID: XKC4:VJXM:XIAR:E7LT:4ELP:G7LS:4KPE:MFIB:T7E4:2YGT:AQCR:IIRE
Docker Root Dir: /data/docker/docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: http://proxy:8080
Https Proxy: http://proxy:8080
No Proxy: .ccveu.local
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Labels:
 debops.ccv.architecture=Ubuntu 16.04
 debops.ccv.environment=production
Insecure Registries:
 127.0.0.0/8

Additional environment details (AWS, VirtualBox, physical, etc.): This is reproducable on the following environments:

• VirtualBox Ubuntu 16.04
• ESX Virtual Machine Ubuntu 16.04
• Virtual Box with Debian Jessie.
• Physical Fedora 19 laptop with docker 1.9.1