traefik: Problem with text/event-stream when compress=true
Do you want to request a feature or report a bug?
Bug
What did you do?
We use concourse (it’s a CI/CD tool) under traefik. Concourse UI uses text/stream
to get information about builds from its backend. In traefik 1.3.8 everything works fine, but after updating to traefik 1.4.0 - 1.4.5 concourse can’t get any event from its backend (infinite waiting…).
What did you expect to see?
Concourse gets events from the backend.
What did you see instead?
Infinite waiting.
Output of traefik version
: (What version of Traefik are you using?)
Traefik 1.4.0 - 1.4.5
What is your environment & configuration (arguments, toml, provider, platform, …)?
The route to concourse was configured statically.
################################################################
# Global configuration
################################################################
logLevel = "DEBUG"
# Traefik logs file
traefikLogsFile = "/var/log/traefik-debug.log"
# Access logs file
accessLogsFile = "/var/log/accessTraefik.log"
# Entrypoints
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
compress = true
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
certFile = ""
keyFile = ""
# ...
[file]
[backends]
[backends.concourse]
[backends.concourse.servers.server1]
url = "http://some-ip:8080"
# ...
[frontends]
[frontends.concourse]
backend = "concourse"
passHostHeader = true
[frontends.concourse.routes.host]
rule = "Host:some-host"
If applicable, please paste the log output in debug mode (--debug
switch)
Only these lines are related to concourse in traefik logs.
time="2017-12-15T22:13:12Z" level=debug msg="Round trip: http://some-ip:port, code: 200, duration: 6.000457ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:some-host"
time="2017-12-15T22:13:14Z" level=debug msg="Round trip: http://some-ip:port, code: 200, duration: 2.51749ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:some-host"
time="2017-12-15T22:13:14Z" level=debug msg="Round trip: http://some-ip:port, code: 200, duration: 3.597858ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:some-host"
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 4
- Comments: 39 (9 by maintainers)
Commits related to this issue
- Do not compress when content type text/event-stream Fix #2576 — committed to sylr/traefik by deleted user 5 years ago
Any update on this? The problem still continues for me on concourse helm chart: concourse-3.0.1 | traefik-1.56.1.
@camelpunch Thank you for sending us the steps to reproduce the issue, but we already knew how to reproduce it thanks to @Overbryd !
The issue is not related to
concourse
at all in fact, it simply happens with the combination of compressed entrypoints and HTTP streams.We know what the issue is, but we have not yet decided how to handle it, we’ll let you know when we do know.
In the meantime, the only workaround is to either disable compression or avoid using streams, as far as I know.
this is crazy… I just reproduced it on a brand new debian:9 box
The exact config files are right here…
Its currently running if you’d like to test although I may switch it to http mode to quickly check
fly -t ci login -c http://bot.deltashim.com
edit: I wonder if because I have docker-compose v 2 there it’ll make a difference?
edit2: I logged in on :80 and it works perfectly there!! but not over https
edit3: omg its working on both now… all I did was some restarts and remove the http>https redirect. With the redirect back in its working on https again… what the heck is going on… going to do some more tests including removing networks/volumes between tests
I’ve noticed when I’ve disabled compression on https endpoint, the issue disappeared
Kubernetes 1.10.8 (minikube) Traefik 1.6.6 (helm chart stable/traefik 1.43.0) Concourse 3.14.1 (helm chart stable/concourse 1.3.1)
Encountered this issue today. Same as in previous posts - stuck on “events” API call. Traefik 1.7.0, Concourse 4.2.1.
I just ran into this issue. For what its worth, I think the size of the payload makes a difference. When I have less than 100 bytes, it shows no data. But when the payload grows bigger then the data shows up.
@Ullaakut all correct. Mind the trailing slash on your request. It should be
curl --compressed localhost:8080/events/
.I have the same with traefik 1.6.2 with acme concourse 3.14.1 both installed from stable Helm charts.
The setup is pretty similar to others, but if I can help somehow (logs, testing some new version) - let me know 😉
@emcniece I was certain I’d be able to reproduce this easily with docker-compose but then I tried and disturbingly it just always worked… I even went the route of host networking for traefik to be sure. I can see why you were so frustrated in your tests
I am pretty sure (when I have the time this week) that I can reproduce it by putting https on the frontend of a docker-compose for u.
The first time I encountered this issue was when I hand-rolled a deployment on a single instance and used docker-compose to flesh out what I needed… naturally I used traefik+acme ssl but I thought it was concourse that was faulty…
It was only later with k8s also that I isolated it the cause to be traefik. I switched ingress controllers and that sorted the issue. (although I’m not happy about a life without traefik)
TL;DR: The only common thread that I can think of now is having https on the frontend
@abellion, I’m not enable to reproduce, if you have a more reproducible case, I can’t try again.
I tried with concourse 3.9.2. I follow https://concoursetutorial.com/ and https://concoursetutorial.com/basics/task-hello-world/ to install my concourse and create a build.
Then I launched a Traefik 1.5.4 (https://github.com/containous/traefik/releases/download/v1.5.4/traefik_linux-amd64) with this config:
And everythings seems to work on
http://127.0.0.1:8081/builds/1
my working Concourse version is 3.8.0. I’m currently running Traefik 1.5.3