nginx-proxy: Can't proxy to containers running in host network mode
When using nginx-proxy to try to proxy to a container running in host networking mode, I assume I also have to run nginx-proxy in host network mode as well (although I’ve tried both ways without success) but I can’t get it to work. Here’s a sample compose file using the “web” image used in the test suite:
version: '2'
services:
nginx-proxy:
image: jwilder/nginx-proxy:test
network_mode: "host"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./lib/ssl/dhparam.pem:/etc/nginx/dhparam/dhparam.pem:ro
web1:
image: web
expose:
- "81"
environment:
WEB_PORTS: 81
VIRTUAL_HOST: web1.nginx-proxy.local
web2:
image: web
expose:
- "82"
network_mode: "host"
environment:
WEB_PORTS: 82
VIRTUAL_HOST: web2.nginx-proxy.local
after running this with docker-compose -f test_network_mode_host.yml up -d I try to curl each:
$ curl localhost:80/port -H "Host: web1.nginx-proxy.local"
answer from port 81
$ curl localhost:80/port -H "Host: web2.nginx-proxy.local"
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.13.8</center>
</body>
</html>
I can, however get to web2 using localhost
curl 127.0.0.1:82/port
answer from port 82
The problem seems to be in the upstream section for web2, which just has server 127.0.0.1 down;
Here’s the full /etc/nginx/conf.d/default.conf:
# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
default $http_x_forwarded_proto;
'' $scheme;
}
# If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
# server port the client connected to
map $http_x_forwarded_port $proxy_x_forwarded_port {
default $http_x_forwarded_port;
'' $server_port;
}
# If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
# Connection header that may have been passed to this server
map $http_upgrade $proxy_connection {
default upgrade;
'' close;
}
# Apply fix for very long server names
server_names_hash_bucket_size 128;
# Default dhparam
ssl_dhparam /etc/nginx/dhparam/dhparam.pem;
# Set appropriate X-Forwarded-Ssl header
map $scheme $proxy_x_forwarded_ssl {
default off;
https on;
}
gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
log_format vhost '$host $remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
access_log off;
resolver 10.0.2.3;
# HTTP 1.1 support
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;
# Mitigate httpoxy attack (see README for details)
proxy_set_header Proxy "";
server {
server_name _; # This is just an invalid value which will never trigger on a real hostname.
listen 80;
access_log /var/log/nginx/access.log vhost;
return 503;
}
# web1.nginx-proxy.local
upstream web1.nginx-proxy.local {
## Can be connect with "test_sneakernet" network
# test_web1_1
server 172.18.0.3:81;
}
server {
server_name web1.nginx-proxy.local;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
location / {
proxy_pass http://web1.nginx-proxy.local;
}
}
# web2.nginx-proxy.local
upstream web2.nginx-proxy.local {
## Can be connect with "host" network
# test_web2_1
server 127.0.0.1 down;
}
server {
server_name web2.nginx-proxy.local;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
location / {
proxy_pass http://web2.nginx-proxy.local;
}
}
Am I missing something in setting this up or is it just not working like it’s supposed to?
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 21
- Comments: 36
Same issue…having trouble getting this to work with Home-Assistant (which needs network_mode host to do some UPnP discovery)
I’m currently skirting around this “bug”/“limitation” by using socat, since I was previously using socat to handle redirection to a raspberry pi with homeassistant on it, but since I have been consolidating a few things I decided to move hass to a container. Similar to the above openhab cases, its beneficial to use host mode networking. Anyway, the short version of my solution is to use the following in one of my
docker-compose.yml:Where homeassistant is listening on the host in another stack on 8123. The socat container here handles the nginx/letsencrypt binding with this project (more or less how I had it working when it was external; however, now it points the host IP of the docker instance and just uses a different port for its nginx virtual host. Works like a charm.
Had the same issue with Home Assistant running on network_mode host and getting the upstream variable to the correct IP and port.
I ended up creating a configuration file in /etc/nginx/conf.d/your.domain.com.conf specific to the host (your.domain.com:8123). Inside my docker-compose file, I did not include Virtual_Host. Mounted the /conf.d volume outside the container as well.
version: ‘3’ services: homeassistant: container_name: homeassistant image: homeassistant/home-assistant:stable privileged: true volumes: - /path/to/configs/hass:/config environment: - PUID=1000 - PGID=1000 - TZ=America/Los_Angeles - LETSENCRYPT_HOST=your.domain.com - LETSENCRYPT_EMAIL=your@email.com network_mode: host
/etc/nginx/conf.d/your.domain.com.conf
your.domain.com
upstream your.domain.com { # Cannot connect to network of this container server 10.0.0.4:8123; #Host IP Address and port } server { server_name your.domain.com; listen 80 ; access_log /var/log/nginx/access.log vhost; # Do not HTTPS redirect Let’sEncrypt ACME challenge location /.well-known/acme-challenge/ { auth_basic off; allow all; root /usr/share/nginx/html; try_files $uri =404; break; } location / { return 301 https://$host$request_uri; } } server { server_name your.domain.com; listen 443 ssl http2 ; access_log /var/log/nginx/access.log vhost; ssl_session_timeout 5m; ssl_session_cache shared:SSL:50m; ssl_session_tickets off; ssl_certificate /etc/nginx/certs/your.domain.com.crt; ssl_certificate_key /etc/nginx/certs/your.domain.com.key; ssl_dhparam /etc/nginx/certs/your.domain.com.dhparam.pem; ssl_stapling on; ssl_stapling_verify on; ssl_trusted_certificate /etc/nginx/certs/your.domain.com.chain.pem; add_header Strict-Transport-Security “max-age=31536000” always; include /etc/nginx/vhost.d/default; location / { proxy_pass http://your.domain.com; } }
This issue seems to have gotten stale, but I am running into this as well trying to get home assistant working properly. Without host networking mode, Hass can’t find things like my Plex server or Google homes.
I can run my nginx in bridge mode and have proxy a container in host mode. However, I’ve had to alter the template as I describe here:
https://github.com/jwilder/nginx-proxy/issues/832
I reported it quite a while ago, but I haven’t heard anything yet on a native solution.
Thanks @kariudo for the example. I modified it to work with non-bridge networks, maybe that’s interesting for @anLizard as well. The secret ingredient is host.docker.internal:
See also https://docs.docker.com/desktop/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host
@anLizard, Rather than clogging up this issue, here is a more complete example of a
docker-compose.ymlfor how I am handling using nginx-proxy to resolve this.https://gist.github.com/kariudo/0e2531ef8165a6f8650cc81df56083a7
I can’t attest to other environments, but I can confirm this works for me quite well.
Thanks @Lif3line and @Kami for the first revision. This works like a charm for OpenHAB also.
I’ve modified nginx.tmpl as you commented
In my case:
ifconfigreturns it asdocker0:ip addressThe docker-compose.yml for openhab is:
That’s all. Maybe this can help other Openhab users.
Thanks @Kami, your suggested solution was excellent. I had the same use-case as others; wanting to run Home Assistant with
network_mode: host. In case anyone wants to replicate:I found
nginxfell over with the original suggested patch since it led to 2serverentries, but that’s only a minor change:The
<docker internal ip>needs to be whatifconfigshows for Docker, normally that’s under the headingdocker0:or similar.I had difficulty rebuilding the reverse proxy image from source as well as using the
docker-composekeywordcommandto insert the patch at start-up, so I settled on building on top of the reverse proxy image:where
hass_fix.patchis the above patch file and must reside in the same directory as thisDockerfile.The process was then:
docker build .on the folder withhass_fix.patchand theDockerfiledocker tag <hash> host_mode_jwilderhost_mode_jwilderimagenetwork_mode: hostVIRTUAL_HOSTandLETSENCRYPT_HOSTenvironment variables@YouDontGitMe I don’t think this will work correctly due to how included config files are handled in nginx (I also tested similar approach / workaround first).
This will only work if you have a single vhost served by nginx (aka
your.domain.com).If you have multiple vhosts, nginx will serve certificate for
your.domain.comfor all others vhosts as well and it won’t work because server block for your.domain.com will have precedence over server blocks indefault.confwhich is generated from template in this repo.By default,
nginx.tmplwill generate an entry like this for container which is usinghostnetworking:But we want something like this:
Right now, my work around includes a custom
Dockerfile+Procfilefornginx-proxyimage which usessedto manipulatedefault.confentry for vhost entry where docker host networking is used.This approach is definitely on the hacky side and there are nicer workarounds possible (e.g. just add some if statements to the template file itself or just add support for environment variables for managing more complex setups), but it works.
Here is my
Dockerfile:Procfile:fix-ha-vhost.sh:EDIT: For completeness sake, here is also a slightly nicer hack which only relies on small change to upstream
nginx.tmpl.I had a similar issue and fixed it using this configuration:
nginx-proxy/docker-compose.yml:
wp1.local/docker-compose.yml
Note the network name. I didn’t create it manually, it is based on the nginx-proxy default network. I’m on a Mac, and after setup the containers I added the following line to the end of my hosts file:
@neographikal Where you able to find a solution for the Home Assistant?
I’m going to chime in as yet another person trying to use Home Assistant with this container. Some services (HomeKit, in my case) don’t work unless the Home Assistant container is running in host networking mode – but doing that completely breaks the reverse proxy.
Same here. I have an OpenHAB container which has to be on the host network, but still I want to have a proxy for authentication.