traefik: letsEncrypt not working with etcd

When I try to configure Traefik in combination with ‘etcd’ as a backend store, i can’t get LetsEncrypt working.

traefik.toml (imported with storeconfig)

[web]
address = ":8080"
defaultEntryPoints = ["https"]
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
[acme]
email = "me@domain.tld"
storage= "/traefik/acme/account"
entryPoint = "https"
onDemand = false
OnHostRule = true
caServer = "https://acme-staging.api.letsencrypt.org/directory"
[[acme.domains]]
   main = "domain.tld"
   sans = ["www.domain.tld", "ww2.domain.tld"]
[etcd]
endpoint = "127.0.0.1:4001"
watch = true
prefix = "/traefik"

Output in the logs

Dec 02 09:23:18 core-01 rkt[21448]: [131075.886503] traefik[5]: time="2016-12-02T10:23:18+01:00" level=info msg="Skipping same configuration for provider etcd"
Dec 02 09:23:22 core-01 rkt[21448]: [131079.650257] traefik[5]: time="2016-12-02T10:23:22+01:00" level=debug msg="Challenge GetCertificate www.mydomain.tld"
Dec 02 09:23:22 core-01 rkt[21448]: [131079.650881] traefik[5]: time="2016-12-02T10:23:22+01:00" level=debug msg="Loading ACME certificates [www.mydomain.tld]..."
Dec 02 09:23:22 core-01 rkt[21448]: [131079.661423] traefik[5]: time="2016-12-02T10:23:22+01:00" level=debug msg="Challenge GetCertificate www.mydomain.tld"
Dec 02 09:23:22 core-01 rkt[21448]: [131079.662657] traefik[5]: time="2016-12-02T10:23:22+01:00" level=debug msg="Loading ACME certificates [www.mydomain.tld]..."
Dec 02 09:23:22 core-01 rkt[21448]: [131080.212813] traefik[5]: time="2016-12-02T10:23:22+01:00" level=debug msg="Challenge Present www.mydomain.tld"
Dec 02 09:23:22 core-01 rkt[21448]: [131080.238591] traefik[5]: time="2016-12-02T10:23:22+01:00" level=debug msg="Challenge Present www.mydomain.tld"

From the output in the logs I can’t really find out what is exactly going on. I had a look at the source code to see where it might go wrong, but my Go knowledge is no-exiting atm.

Slack reference : https://traefik.slack.com/archives/support/p1480674299005971

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 11
Comments: 26 (3 by maintainers)

Commits related to this issue

[vendor] use abronan/libkv instead of docker/libkv; docker/libkv is dead and abronan/libkv is the repository where development is happening. this solves #1975 and therefore hopefully #926 once the et... — committed to trusch/traefik by trusch 7 years ago

Most upvoted comments

I’m catching up on this issue, trying to slowly get the libkv repository back on track. In the meantime I’ll start working on the etcd v3 backend which should hopefully fix the problem (I see it has a proper Mutex API now which was not the case when I originally opened docker/libkv#103 and docker/libkv#106).

As @lorenz mentioned above, the interface probably maps badly with the new lock implementation but I’ll be aiming for something that works at first. We can later discuss about a possible interface change to better map with new primitives.

I’ll update as soon as I have some results.

abronan on Aug 17, 2017

This seems to still be an issue. Tried with traefik 1.7.4 and an etcd-3.3.9.

KlavsKlavsen on Nov 7, 2018

Let me know if everything works out fine or if there are still remaining issues and I’ll take a look. Feel free to open an issue on the fork as well. I may create a new repository as some have issues with import paths and the fact that it still points to the docker/libkv repository (some package managers support aliasing, some not). But we’ll migrate issues if we go forward with that approach.

abronan on Aug 29, 2017

Update: I got the Lock working with the v3 client (using the same model as for zookeeper: a side-key for the lock). It works as intended and it’s pretty clean now compared to v2, just a few lines of code.

What’s left:

Finish, Polish and Test the v3 backend: I’m missing calls like Watch/WatchTree and List/DeleteTree.
I realized I can also probably fix the v2 custom lock feature using the same method with a side-key (Yay!)

Will update again as soon as I drop a PR.

abronan on Aug 18, 2017

everything is merged into abronan/libkv! I hope this fixes will be integrated in near future 😃

trusch on Aug 29, 2017

Just want to add that this appears to be happening on zookeeper as well:

ERRO[2017-03-15T00:31:49Z] Datastore sync error: Object lock value: expected c6ff697f-3880-422d-9e06-2e44bb8ce18a, got , retrying in 588.410415ms 
ERRO[2017-03-15T00:31:50Z] Datastore sync error: Object lock value: expected c6ff697f-3880-422d-9e06-2e44bb8ce18a, got , retrying in 738.289049ms 
ERRO[2017-03-15T00:31:51Z] Datastore sync error: Object lock value: expected c6ff697f-3880-422d-9e06-2e44bb8ce18a, got , retrying in 1.151435232s 
ERRO[2017-03-15T00:31:52Z] Datastore sync error: Object lock value: expected c6ff697f-3880-422d-9e06-2e44bb8ce18a, got , retrying in 1.965909627s 
ERRO[2017-03-15T00:31:54Z] Datastore sync error: Object lock value: expected c6ff697f-3880-422d-9e06-2e44bb8ce18a, got , retrying in 2.138171143s 
ERRO[2017-03-15T00:31:56Z] Datastore sync error: Object lock value: expected c6ff697f-3880-422d-9e06-2e44bb8ce18a, got , retrying in 5.3727141s

EDIT: Appears to be unrelated making new issue

andrewl3wis on Mar 15, 2017