traefik: kv value too large when using consul backend and acme
What version of Traefik are you using (traefik version)?
v.1.1.2
What is your environment & configuration (arguments, toml…)?
docker swarm 1.13
Containers: 5
Running: 2
Paused: 0
Stopped: 3
Images: 33
Server Version: 1.13.0
Storage Driver: overlay
Backing Filesystem: extfs
Supports d_type: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: glusterfs local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: c8rqvh23p2kqqpbyj9ydf5xxy
Is Manager: true
ClusterID: n1td1rbeiwcokhp7gglg6z4gy
Managers: 3
Nodes: 3
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 10.135.25.157
Manager Addresses:
10.135.25.157:2377
10.135.36.196:2377
10.135.8.136:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-59-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.859 GiB
Name: ohmy-rancher1
ID: ZC23:CHZC:AT2V:KFGG:AD7T:NPUZ:H5SY:BA6N:3KIT:ORRK:X6HW:O7PZ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
127.0.0.0/8
traefik is running on all three nodes as load balancer, output below is not perfectly valid json as I cut out the PreviousSpec and UpdateStatus bits.
docker service inspect traefik
[
{
"ID": "r2i408yp9lj1t319voglwd4ay",
"Version": {
"Index": 146
},
"CreatedAt": "2017-01-27T09:38:04.005237112Z",
"UpdatedAt": "2017-01-27T09:53:47.215408631Z",
"Spec": {
"Name": "traefik",
"TaskTemplate": {
"ContainerSpec": {
"Image": "traefik:latest@sha256:c81e1a321ec90a41987b1c8cb887606e3e632acadb3b29f5b97f0ad03c59ffd9",
"Args": [
"--web",
"--consul",
"--consul.endpoint=consul.service.consul:8500",
"--docker",
"--docker.swarmmode",
"--docker.watch"
],
"Mounts": [
{
"Type": "bind",
"Source": "/var/run/docker.sock",
"Target": "/var/run/docker.sock"
}
],
"DNSConfig": {}
},
"Resources": {
"Limits": {},
"Reservations": {}
},
"RestartPolicy": {
"Condition": "any",
"MaxAttempts": 0
},
"Placement": {
"Constraints": [
"node.role==manager"
]
},
"Networks": [
{
"Target": "s7xhhuwbexlefkq87eu1jyvvb"
},
{
"Target": "me7eoua9vuvrg1ywy3p3sm1ik"
}
],
"ForceUpdate": 0
},
"Mode": {
"Global": {}
},
"UpdateConfig": {
"Parallelism": 1,
"FailureAction": "pause",
"MaxFailureRatio": 0
},
"Networks": [
{
"Target": "traefik"
},
{
"Target": "ingress"
}
],
"EndpointSpec": {
"Mode": "vip",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 80,
"PublishedPort": 80,
"PublishMode": "ingress"
},
{
"Protocol": "tcp",
"TargetPort": 8080,
"PublishedPort": 8080,
"PublishMode": "ingress"
},
{
"Protocol": "tcp",
"TargetPort": 443,
"PublishedPort": 443,
"PublishMode": "ingress"
}
]
}
},
"Endpoint": {
"Spec": {
"Mode": "vip",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 80,
"PublishedPort": 80,
"PublishMode": "ingress"
},
{
"Protocol": "tcp",
"TargetPort": 8080,
"PublishedPort": 8080,
"PublishMode": "ingress"
},
{
"Protocol": "tcp",
"TargetPort": 443,
"PublishedPort": 443,
"PublishMode": "ingress"
}
]
},
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 80,
"PublishedPort": 80,
"PublishMode": "ingress"
},
{
"Protocol": "tcp",
"TargetPort": 8080,
"PublishedPort": 8080,
"PublishMode": "ingress"
},
{
"Protocol": "tcp",
"TargetPort": 443,
"PublishedPort": 443,
"PublishMode": "ingress"
}
],
"VirtualIPs": [
{
"NetworkID": "me7eoua9vuvrg1ywy3p3sm1ik",
"Addr": "10.255.0.6/16"
},
{
"NetworkID": "s7xhhuwbexlefkq87eu1jyvvb",
"Addr": "172.20.0.2/16"
}
]
}
per@host:~$ consul kv get --recurse traefik
traefik/accesslogsfile:
traefik/acme/:
traefik/acme/account/lock:
traefik/acme/account/object:REMOVED
traefik/acme/caserver:
traefik/acme/email:REMOVED
traefik/acme/entrypoint:https
traefik/acme/ondemand:true
traefik/acme/onhostrule:true
traefik/acme/storage:traefik/acme/account
traefik/checknewversion:true
traefik/consul/:
traefik/consul/endpoint:consul.service.consul:8500
traefik/consul/filename:
traefik/consul/prefix:traefik
traefik/consul/watch:true
traefik/debug:false
traefik/defaultentrypoints/0:http
traefik/defaultentrypoints/1:https
traefik/docker/:
traefik/docker/domain:docker.localhost
traefik/docker/endpoint:unix:///var/run/docker.sock
traefik/docker/exposedbydefault:true
traefik/docker/filename:
traefik/docker/swarmmode:true
traefik/docker/usebindportip:false
traefik/docker/watch:true
traefik/entrypoints/http/:
traefik/entrypoints/http/address::80
traefik/entrypoints/http/compress:false
traefik/entrypoints/http/network:
traefik/entrypoints/https/:
traefik/entrypoints/https/address::443
traefik/entrypoints/https/compress:false
traefik/entrypoints/https/network:
traefik/entrypoints/https/tls/:
traefik/entrypoints/https/tls/ciphersuites/0:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
traefik/entrypoints/https/tls/ciphersuites/1:TLS_RSA_WITH_AES_256_GCM_SHA384
traefik/entrypoints/https/tls/minversion:VersionTLS12
traefik/gracetimeout:10
traefik/insecureskipverify:false
traefik/leader:b5850a68-1b48-45e3-94c0-edc3d6eb03ae
traefik/loglevel:DEBUG
traefik/maxidleconnsperhost:200
traefik/providersthrottleduration:2000000000
traefik/web/:
traefik/web/address::8080
traefik/web/certfile:
traefik/web/keyfile:
traefik/web/readonly:false
What did you do?
automatically generated a new certificate through acme
What did you expect to see?
certificate generated correctly
What did you see instead?
error presenting token: StoreConfig error: Unexpected response code: 413 (Value exceeds 524288 byte limit)]
Searching for this I came across this commit https://github.com/hashicorp/consul/commit/8f37f967e0fc677fa54c9beea988dc331e35718f which mentiones the 512kb limit.
My thoughts about a solution is along the lines of serializing the acme account data into keys and values instead of json so they use the kv store “properly”.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 7
- Comments: 27 (8 by maintainers)
Encoding with something like MessagePack instead of JSON could also help reducing the size of the payload. https://msgpack.org/index.html
Also, what’s the reason behind storing the payload in base64 if it’s not binary data ? Base64 increase the encoded payload size by 33%.
I’ll hit you up on slack to discuss this a bit further - I’m not sure our use-case is going to match well with the majority of users’ here. Will copy discussion here for discoverability later.