docksal: vhosts config missing upstream for all projects
Description
I have been using Docksal for quite some time on Manjaro Linux without hassle and it was still working fine yesterday. Today however all my projects result in a 502 Bad Gateway from the vhost proxy (browser and curl), entering a Docksal URL for a project that doesn’t exist results in the “Project missing” page so the proxy seems to be running fine. Everything else seems fine as well (all containers running, I can execute drush commands, …).
I did some (minor) updates in Manjaro yesterday so I was assuming that triggered it, one of the packages was docker (20.10.7) so I downgraded it again but to no avail. In the meantime I’ve fully reinstalled docker and docksal but I’m still getting the 502 error.
I did some digging already and discovered that the vhosts.conf file in the proxy container is missing the upstream for the web container, it only contains the “server localhost:80 down;” rule. If I paste the IP for the web container manually and reload the openresty config, the site works fine again, but of course the config disappears again when switching projects.
So now I’m trying to figure out why the rule isn’t appearing, any ideas? Or can you shine some light on how the rule is generated?
Thanks!
Steps to reproduce the issue:
- Install Docksal on Manjaro (might not be OS specific though)
- Set up a project
- Navigate to the project URL using a browser or curl
Describe the results you received:
A 502 Bad Gateway error from openresty / the vhosts proxy.
Describe the results you expected:
A working page from my project.
Output of fin config:
fin config output
---------------------
COMPOSE_PROJECT_NAME_SAFE: sandbox
COMPOSE_FILE:
/home/kobe/.docksal/stacks/volumes-bind.yml
/home/kobe/.docksal/stacks/stack-default.yml
/home/kobe/development/sandbox/.docksal/docksal.yml
ENV_FILE:
/home/kobe/development/sandbox/.docksal/docksal.env
PROJECT_ROOT: /home/kobe/development/sandbox
DOCROOT: web
VIRTUAL_HOST: sandbox.docksal
VIRTUAL_HOST_ALIASES: *.sandbox.docksal
IP: 192.168.64.100
MYSQL: 192.168.64.100:33061
Docker Compose configuration
---------------------
services:
cli:
dns:
- 192.168.64.100
- 8.8.8.8
environment:
BLACKFIRE_CLIENT_ID: 60750*****4417a
BLACKFIRE_CLIENT_TOKEN: f374e*****e6f80
COMPOSER_ALLOW_XDEBUG: '1'
COMPOSER_DEFAULT_VERSION: null
COMPOSER_DISABLE_XDEBUG_WARN: '1'
DOCROOT: web
DRUSH_ALLOW_XDEBUG: '1'
DRUSH_OPTIONS_URI: sandbox.docksal
GIT_USER_EMAIL: ''
GIT_USER_NAME: Kobe Wright
HOST_GID: '1000'
HOST_UID: '1000'
MYSQL_DATABASE: docksal
MYSQL_HOST: db
MYSQL_PASSWORD: docksal
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: docksal
PHP_IDE_CONFIG: null
SECRET_ACAPI_EMAIL: null
SECRET_ACAPI_KEY: null
SECRET_ACQUIACLI_KEY: null
SECRET_ACQUIACLI_SECRET: null
SECRET_PLATFORMSH_CLI_TOKEN: null
SECRET_SSH_PRIVATE_KEY: null
SECRET_TERMINUS_TOKEN: null
SSH_AUTH_SOCK: /.ssh-agent/proxy-socket
VIRTUAL_HOST: sandbox.docksal
XDEBUG_CONFIG: remote_connect_back=0 remote_host=192.168.64.1
XDEBUG_ENABLED: '1'
healthcheck:
interval: 10s
hostname: cli
image: docksal/cli:2.11-php7.3
labels:
io.docksal.shell: bash
io.docksal.user: docker
logging:
options:
max-file: '10'
max-size: 1m
ports:
- published: 3000
target: 3000
- published: 3001
target: 3001
- published: 8080
target: 8080
volumes:
- docksal_ssh_agent:/.ssh-agent:ro
- cli_home:/home/docker:rw
- /tmp/.docksal/sandbox:/tmp/.docksal/sandbox:ro
- project_root:/var/www:rw,nocopy,cached
db:
dns:
- 192.168.64.100
- 8.8.8.8
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: null
MYSQL_DATABASE: docksal
MYSQL_INITDB_SKIP_TZINFO: null
MYSQL_ONETIME_PASSWORD: null
MYSQL_PASSWORD: docksal
MYSQL_RANDOM_ROOT_PASSWORD: null
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: docksal
healthcheck:
interval: 10s
hostname: db
image: docksal/mysql:5.6-1.5
logging:
options:
max-file: '10'
max-size: 1m
ports:
- published: 33061
target: 3306
volumes:
- db_data:/var/lib/mysql:rw
- project_root:/var/www:ro,nocopy,cached
mail:
dns:
- 192.168.64.100
- 8.8.8.8
healthcheck:
interval: 10s
hostname: mail
image: mailhog/mailhog
labels:
io.docksal.cert-name: none
io.docksal.virtual-host: mail.sandbox.docksal,mail.sandbox.docksal.*
io.docksal.virtual-port: '8025'
logging:
options:
max-file: '10'
max-size: 1m
volumes:
- project_root:/var/www:ro,nocopy,cached
redis:
dns:
- 192.168.64.100
- 8.8.8.8
environment:
REDIS_MAXMEMORY: 256m
healthcheck:
interval: 10s
hostname: redis
image: wodby/redis:5.0
logging:
options:
max-file: '10'
max-size: 1m
solr:
image: docksal/solr:8-2.1
ports:
- published: 8983
target: 8983
volumes:
- /home/kobe/development/sandbox/.docksal/etc/solr/conf:/opt/solr/server/solr/conf:ro
varnish:
depends_on:
web:
condition: service_started
dns:
- 192.168.64.100
- 8.8.8.8
environment:
VARNISH_BACKEND_HOST: web
healthcheck:
interval: 10s
hostname: varnish
image: docksal/varnish:6.1-2.0
labels:
io.docksal.cert-name: none
io.docksal.virtual-host: varnish.sandbox.docksal,varnish.sandbox.docksal.*
logging:
options:
max-file: '10'
max-size: 1m
volumes:
- project_root:/var/www:ro,nocopy,cached
web:
depends_on:
cli:
condition: service_started
dns:
- 192.168.64.100
- 8.8.8.8
environment:
APACHE_BASIC_AUTH_PASS: null
APACHE_BASIC_AUTH_USER: null
APACHE_DOCUMENTROOT: /var/www/web
APACHE_FCGI_HOST_PORT: cli:9000
healthcheck:
interval: 10s
hostname: web
image: docksal/apache:2.4-2.3
labels:
io.docksal.cert-name: none
io.docksal.permanent: "false"
io.docksal.project-root: /home/kobe/development/sandbox
io.docksal.virtual-host: sandbox.docksal,*.sandbox.docksal,sandbox.docksal.*
logging:
options:
max-file: '10'
max-size: 1m
volumes:
- project_root:/var/www:ro,nocopy,cached
version: '2.1'
volumes:
cli_home: {}
db_data: {}
docksal_ssh_agent:
external: true
name: docksal_ssh_agent
project_root:
driver: local
driver_opts:
device: /home/kobe/development/sandbox
o: bind
type: none
---------------------
Output of fin sysinfo:
fin sysinfo output
███ FIN
fin version: 1.103.0
███ OS
Linux Manjaro Linux
Linux desktop-kobe-manjaro 5.9.16-1-MANJARO #1 SMP PREEMPT Mon Dec 21 22:00:46 UTC 2020 x86_64 GNU/Linux
███ ENVIRONMENT
MODE : Linux Kernel
DOCKER_HOST :
███ DOCKER
EXPECTED CLIENT VERSION: 19.03.13
EXPECTED SERVER VERSION: 19.03.13
Client:
Version: 20.10.6
API version: 1.41
Go version: go1.16.3
Git commit: 370c28948e
Built: Mon Apr 12 14:10:41 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.6
API version: 1.41 (minimum version 1.12)
Go version: go1.16.3
Git commit: 8728dd246c
Built: Mon Apr 12 14:10:25 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.5.2
GitCommit: 36cc874494a56a253cd181a1a685b44b58a2e34a.m
runc:
Version: 1.0.0-rc95
GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
docker-init:
Version: 0.19.0
GitCommit: de40ad0
███ DOCKER COMPOSE
EXPECTED VERSION: 1.27.4
docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019
███ DOCKSAL: PROJECTS
project STATUS virtual host project root
sandbox Up 2 hours (healthy) sandbox.docksal,*.sandbox.docksal,sandbox.docksal.* /home/kobe/development/sandbox
███ DOCKSAL: VIRTUAL HOSTS
*.sandbox.docksal
sandbox.docksal.*
sandbox.docksal
mail.sandbox.docksal.*
mail.sandbox.docksal
varnish.sandbox.docksal.*
varnish.sandbox.docksal
███ DOCKSAL: NETWORKING
DOCKSAL_IP: 192.168.64.100
DOCKSAL_HOST_IP: 192.168.64.1
DOCKSAL_VHOST_PROXY_IP:
DOCKSAL_DNS_IP:
DOCKSAL_DNS_DISABLED: 0
DOCKSAL_NO_DNS_RESOLVER: 0
DOCKSAL_DNS_UPSTREAM:
DOCKSAL_DNS_DOMAIN: docksal
███ DOCKSAL: CONNECTIVITY
Host to 192.168.64.100: PASS
Container to 192.168.64.100: PASS
Container to 192.168.64.1: PASS
Checking connectivity to http://dns-test.docksal...
Host: PASS
Containers: PASS
███ DOCKER: RUNNING CONTAINERS
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a45a93939d74 docksal/varnish:6.1-2.0 "docker-entrypoint.s…" 2 hours ago Up 2 hours (healthy) 80/tcp, 6082/tcp sandbox_varnish_1
12ec06e5a7f7 docksal/apache:2.4-2.3 "httpd-foreground" 2 hours ago Up 2 hours (healthy) 80/tcp, 443/tcp sandbox_web_1
ef02d16eec50 wodby/redis:5.0 "/docker-entrypoint.…" 2 hours ago Up 2 hours 6379/tcp sandbox_redis_1
b8e568988dc8 docksal/cli:2.11-php7.3 "/opt/startup.sh sup…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:3000-3001->3000-3001/tcp, :::3000-3001->3000-3001/tcp, 22/tcp, 9000/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp sandbox_cli_1
37741037e751 mailhog/mailhog "MailHog" 2 hours ago Up 2 hours 1025/tcp, 8025/tcp sandbox_mail_1
48d795ee724a docksal/solr:8-2.1 "docker-entrypoint.s…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:8983->8983/tcp, :::8983->8983/tcp sandbox_solr_1
a33399b6ef88 docksal/mysql:5.6-1.5 "docker-entrypoint.s…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:33061->3306/tcp, :::33061->3306/tcp sandbox_db_1
219ec5b9e036 docksal/ssh-agent:1.3 "docker-entrypoint.s…" 3 hours ago Up 3 hours (healthy) docksal-ssh-agent
769ec23bb685 docksal/dns:1.1 "docker-entrypoint.s…" 3 hours ago Up 3 hours (healthy) 192.168.64.100:53->53/udp docksal-dns
b052e0337e0b docksal/vhost-proxy:1.6 "docker-entrypoint.s…" 3 hours ago Up 2 hours (healthy) 192.168.64.100:80->80/tcp, 192.168.64.100:443->443/tcp docksal-vhost-proxy
███ DOCKER: NETWORKS
NETWORK ID NAME DRIVER SCOPE
d2e0ac1a3b8b _default bridge local
f29b3edad0f3 bridge bridge local
90eb5b2fc0c9 sandbox_default bridge local
5e5598a0a204 host host local
dc067c7553d1 none null local
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 3
- Comments: 18 (8 by maintainers)
Some good news 🙂 I was able to fix it by updating docker-gen to 0.7.6 in the vhost proxy container so it definitely seems related to the issue I mentioned earlier (I saw some fixes in ).
I did the following:
docker exec docksal-vhost-proxy sh -c "curl -sSL -O https://github.com/nginx-proxy/docker-gen/releases/download/0.7.6/docker-gen-alpine-linux-amd64-0.7.6.tar.gz && tar -C /usr/local/bin -xvzf docker-gen-alpine-linux-amd64-0.7.6.tar.gz"docker restart docksal-vhost-proxyfin p restartI saw that version 0.7.6 is already being used in the develop branch of https://github.com/docksal/service-vhost-proxy so maybe it’s a good idea to create a tag in the near future?
Edit: I also updated to Docker 20.10.7 again since I switched to the develop branch of Docksal, so 20.10.7 is not an issue it seems.
Resolved in docksal/vhost-proxy:v1.7, which will be used in the upcoming Docksal release.
You can switch to that image right away using the following command:
I’m in exactly the same situation, kinda reassuring to hear that it’s not just me!
It does seem specific to Docksal too, I have other non-Docksal projects that depend on Docker and they all seem to still be working just fine.