traefik: websocket errors after upgrading traefik from 3.2.3 to 3.2.4 or 3.3.0
Welcome!
- Yes, I’ve searched similar issues on GitHub and didn’t find any.
- Yes, I’ve searched similar issues on the Traefik community forum and didn’t find any.
What did you do?
I am running MinIO behind traefik using docker compose, and after upgrading today to 3.2.4 or 3.3.0 MinIO fails with websocket errors when trying to list/browse objects in a bucket through the console web interface. Downgrading to 3.2.3 fixes it again with the following compose file listed below. What broke!?!
What did you see instead?
websocket connection error in browser developer console and nothing is listed…
What version of Traefik are you using?
3.2.4
What is your environment & configuration?
services:
minio1:
image: minio/minio:RELEASE.2024-12-18T13-15-44Z
restart: unless-stopped
volumes:
- /opt/minio:/opt/minio
expose:
- "9000"
- "9001"
environment:
MINIO_ROOT_USER: ${MINIO_ADMIN_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ADMIN_PASS}
MINIO_VOLUMES: "/opt/minio"
MINIO_PROMETHEUS_AUTH_TYPE: public
command:
- server
- --address
- ":9000"
- --console-address
- ":9001"
healthcheck:
test: [ "CMD", "mc", "ready", "local" ]
interval: 5s
timeout: 5s
retries: 5
labels:
- "traefik.enable=true"
## minio-console
- "traefik.http.routers.minio-console.rule=Host(`minio-console-osl.example.com`)"
- "traefik.http.routers.minio-console.entrypoints=web,websecure"
- "traefik.http.routers.minio-console.tls=true"
- "traefik.http.routers.minio-console.service=minio-console"
- "traefik.http.services.minio-console.loadbalancer.server.port=9001"
## minio-api
- "traefik.http.routers.minio-api.rule=HostRegexp(`[host:.+]`)"
- "traefik.http.routers.minio-api.entrypoints=web,websecure"
- "traefik.http.routers.minio-api.tls=true"
- "traefik.http.routers.minio-api.service=minio-api"
- "traefik.http.services.minio-api.loadbalancer.server.port=9000"
traefik:
# image: "traefik:v3.2.3"
image: "traefik:v3.2.4"
container_name: "traefik"
restart: unless-stopped
command:
- "--log.level=DEBUG"
- "--api"
- "--api.insecure=true"
- "--api.dashboard=true"
- "--accesslog=true"
- "--accesslog.format=json"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--entrypoints.traefik.address=:8080"
- "--providers.file.directory=/traefik-conf/"
- "--providers.file.watch=true"
- "--metrics.prometheus=true"
- "--metrics.prometheus.addEntryPointsLabels=true"
- "--metrics.prometheus.addServicesLabels=true"
- "--entrypoints.web.http.redirections.entryPoint.to=websecure"
- "--entrypoints.web.http.redirections.entryPoint.scheme=https"
- "--entrypoints.web.http.redirections.entrypoint.permanent=true"
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- $PWD/traefik-conf/dynamic-conf.toml:/traefik-conf/dynamic-conf.toml
- $PWD/ssl:/ssl
- /var/log/:/var/log/
depends_on:
- minio1
If applicable, please paste the log output in DEBUG level
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 41
- Comments: 30 (8 by maintainers)
Hey there,
We have found the root cause of the issue. We’re in the process of fixing it and releasing new versions.
In the meantime, you can use the latest version of Traefik if you set the following environment variable:
GODEBUG=http2xconnect=0.We’ll update the issue with the PR that fixes it.
Same goes for HomeAssistant, could not access it through my Traefik URL but local IP worked fine. After downgrading to 3.2.3 it works fine again.
+1 same issue w/ home-assistant
FYI v3.3.1 is now available in docker registry.
Not trying to be too impatient here, but I see no updated docker images, neither on Docker Hub or GHCR. The
GODEBUG=http2xconnect=0workaround works, but it’d be nice to have an updated image that doesn’t need the workaround, if possible 🙏Hey there,
Thanks for raising this issue. We are analyzing it.
We’ll make sure to keep you updated following up on this ticket as soon as we have more information to share.
The Official Docker images are built and published by Docker itself (this is not a classic Docker Hub image).
The images were built an hour ago, but the deployment takes some time.
https://github.com/docker-library/meta/actions/runs/12661032071
You have to wait a bit.
The tag
v3.3has been deployed 6 minutes ago,latestandv38 minutes ago, I think the other will come soon.Edit: all the tags are available now.
In that case, updating those tags must be waiting in a queue somewhere, because those tags are currently definitely pointing at 3.3.0.
You can see it by going to https://hub.docker.com/_/traefik/tags, make a note of the digest for v3.3.0 linux/amd64 (
8fb30ac...), then look at the digest voor v3 linux/amd64, which is identical. Also, thev3tag was pushed hours before thev3.3.1tag was.None of this is your fault though. We’ll just have to wait a little longer.
EDIT: Oh, and none of the tags pushed 2 hours ago actually point at anything, so 3.3.1, 3.2.5 and 2.11.18 can’t currently be pulled at all.
Excellent, thanks! I don’t know if you made that happen, but the various
v3tags (v3,v3.3,3.3and3) still point at the v3.3.0 image. I’d be grateful if those could also be updated.FYI, this seems to also affect
2.11.17: rolled back to2.11.16in the mean timeEDIT: duplicate of above https://github.com/traefik/traefik/issues/11405#issuecomment-2574220527
Closed by #11408.
3.3.0-rc2 was working correctly so must have been a regression introduced afterwards
I’m having the same issue after an upgrade to 2.11.17, rolling back to 2.11.16 fixes Home Assistant for me.
Confirmed. 3.2.4 and 3.3.0 broke my homelab. Back to 3.2.3 waiting for the fix.