docker-socket-proxy: Recurrent logs from Traefik "Provider connection error unexpected EOF"
Hi, we are using docker-socket-proxy with Traefik to protect our docker environment. In traefik v2.0 logs there are many error logs like these :
time="2019-09-30T14:10:37+02:00" level=error msg="Provider connection error unexpected EOF, retrying in 263.019463ms" providerName=docker
time="2019-09-30T14:20:38+02:00" level=error msg="Provider connection error unexpected EOF, retrying in 372.070307ms" providerName=docker
time="2019-09-30T14:30:38+02:00" level=error msg="Provider connection error unexpected EOF, retrying in 726.287014ms" providerName=docker
the docker-socket-proxy configuration is the minimum
- CONTAINERS=1
Do you think theses errors appear because docker-socket-proxy is compatible with Docker API version 1.37 and we are using version 1.40?
Another question, does docker-socket-proxy is still maintained?
Thanks you.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 12
- Comments: 23 (7 by maintainers)
Yes, this is till maintained, although it’s quite stable and has not much patches lately.
It might be related with the api version, maybe we need to update this image for newer versions…
Also I didn’t test Traefik v2 yet.
It’s not an error/issue, it’s just the timeout on the stream of events, if there’s no docker events the connection times out and reconnects.
I think the best fix would be in traefk’s side, lowering that error to warning, and letting users to use
--log-level=error. After all, if Traefik reconnects, nothing bad happens, so it’s technically a warning. It’d be an error if Traefik failed to reconnect.I think there’s nothing we can do downstream, so I close the issue. You can continue discussion, though.
Verifying what @clownba0t said.
I’ve narrowed this down to timeout server in the haproxy.cfg.
Traefik does a request against the events API which blocks and returns events as they happen, if you aren’t getting any docker events in 10 minutes the connection times out and restarts.
So as it stands it isn’t an error since Traefik will just reconnect in less than a second.
I upped the timeout to 1h in my own config, but it just means the error is now every hour instead of every 10 minutes since the containers are pretty stable and there’s no events even in an hour.
The cleanest option would be to use TCP keepalives and option clitcpka/option srvtcpka while disabling haproxy timeouts, but since modifying the TCP keepalive settings inside the docker container isn’t feasible (either running the container as privileged or manually specifying
--sysctloptions), we’re stuck with10mtimeouts.I don’t think converting to Traefik would solve the issue, it has an idleTimeout default of
360s, so that message would be in the logs every 6 minutes instead of 10 minutes, also Traefik doesn’t support unix socket backends.It boils down to how can you tell a connection is still there if it is open but has no traffic.
One idea that floats over my mind is to use a different traefik to proxy the docker socket, instead of HAProxy.
I’m not sure if that’d fix this, but for sure most of our user base uses traefik and would be more able to contribute patches.