etcd: HTTPS only etcd cluster not working (malformed HTTP response)

I tried to set up an etcd cluster with https only addresses. My etcd.service file looks like this:

[Unit]
Description=etcd key-value store
Documentation=https://github.com/coreos/etcd

[Service]
User=root
Type=notify
ExecStart=/usr/local/bin/etcd \
 --name etcd0 \
 --data-dir /var/lib/etcd \
 --listen-peer-urls https://0.0.0.0:2380 \
 --listen-client-urls https://0.0.0.0:2379 \
 --advertise-client-urls https://172.30.102.53:2379,https://172.30.102.53:4001 \
 --initial-advertise-peer-urls https://172.30.102.53:2380 \
 --initial-cluster-token etcd-cluster-1 \
 --initial-cluster etcd0=https://172.30.102.53:2380,etcd1=https://172.30.102.39:2380,etcd2=https://172.30.102.57:2380 \
 --initial-cluster-state existing \
 --heartbeat-interval 1000 \
 --election-timeout 5000 \
 --peer-client-cert-auth \
 --client-cert-auth \
 --cert-file /etc/ssl/etcd/etcd.pem \
 --peer-cert-file /etc/ssl/etcd/etcd.pem \
 --peer-key-file /etc/ssl/etcd/etcd.key \
 --key-file=/etc/ssl/etcd/etcd.key \
 --trusted-ca-file=/etc/ssl/etcd/CA.pem \
 --peer-trusted-ca-file=/etc/ssl/etcd/CA.pem
Restart=always
RestartSec=10s
LimitNOFILE=40000

[Install]
WantedBy=multi-user.target

My output when starting the service is:

ClientTLS: cert = /etc/ssl/etcd/etcd.pem, key = /etc/ssl/etcd/etcd.key, ca = , trusted-ca = /etc/ssl/etcd/CA.pem, client-cert-auth = true, crl-file =
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.57:48228" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.57:48226" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.39:44712" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44714" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.57:48238" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.57:48236" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44720" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44722" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.57:48244" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.57:48246" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44730" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44732" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.39:44734" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.57:48252" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.57:48254" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.39:44740" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44742" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.57:48260" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.57:48262" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.39:44748" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44750" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40  etcd[19990]: rejected connection from "172.30.102.57:48268" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.57:48270" (error "tls: first record does not look like a TLS handshake", ServerName "")
Sep 27 14:35:40 etcd[19990]: rejected connection from "172.30.102.39:44756" (error "tls: first record does not look like a TLS handshake", ServerName "")

… and so on … Between those messages it also says:

publish error: etcdserver: request timed out
Sep 27 14:40:31 etcd[20359]: health check for peer 179451360aef27e2 could not connect: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x
Sep 27 14:40:31 etcd[20359]: health check for peer 4ee5b5f533b5a26e could not connect: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x
Sep 27 14:40:31systemd[1]: etcd.service start operation timed out. Terminating.
Sep 27 14:40:31 systemd[1]: Failed to start etcd key-value store.

I already checked the certificates using “openssl s_client -showcerts -connect [IP]:2380 -cert etcd.pem -key etcd.key -CAfile CA.pem” which seems so be working fine. I’m using the following extensions config:

[ ca ]
# X509 extensions for a ca
keyUsage                = critical, cRLSign, keyCertSign
basicConstraints        = CA:TRUE, pathlen:0
subjectKeyIdentifier    = hash
authorityKeyIdentifier  = keyid:always,issuer:always

[ server ]
# X509 extensions for a server
keyUsage                = critical,digitalSignature,keyEncipherment
extendedKeyUsage        = serverAuth,clientAuth
basicConstraints        = critical,CA:FALSE
subjectKeyIdentifier    = hash
authorityKeyIdentifier  = keyid,issuer:always

I really can’t tell what the problem is. Any ideas?

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 16 (4 by maintainers)

Most upvoted comments

I found the problem, the certificate extensions were not in the certificate (key usage, extended key usage and alternative subject names) - after adding them it worked fine

Could you show me how to add them in detail? Thanks!

How can I add (key usage, extended key usage and alternative subject names) use cfssl? Could you show me ? @Issac-ZY