etcd: The etcd v3 API isn't returning helpful TLS error messages.

Issue

The etcd v3 API isn’t returning helpful TLS error messages. Instead, it simply returns Error: context deadline exceeded. It shoud return TLS errors that are more like the ones returned by the v2 API.

etcdctl version: 3.3.6

This V3 API command:

ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/etcd/ssl/ca.pem --cert=/etc/etcd/ssl/cert.pem --key=/etc/etcd/ssl/key.pem member list

Returns:

Error: context deadline exceeded

While the corresponding command on the V2 API:

ETCDCTL_API=2 etcdctl --endpoints=https://127.0.0.1:2379 --ca-file=/etc/etcd/ssl/ca.pem --cert-file=/etc/etcd/ssl/cert.pem --key-file=/etc/etcd/ssl/key.pem member list

Returns:

client: etcd cluster is unavailable or misconfigured; error #0: x509: certificate is valid for 10.0.0.26, 10.100.0.1, not 127.0.0.1

With the vague error returned by the V3 API, it was impossible to solve the problem. The V2 error message lead me to the issue immediately.

Steps to Reproduce

Download and Build etcd

Follow the instructions here to download and build etcd.

Create Some Test Certs

cd hack/tls-setup

Edit the config/req-csr.json file to have the below JSON (note that the JSON is just the example from hack/tls-setup/README.md with the loopback interface (127.0.0.1) removed).

{
  "CN": "etcd",
  "hosts": [
    "3.8.121.201",
    "46.4.19.20"
  ],
  "key": {
    "algo": "ecdsa",
    "size": 384
  },
  "names": [
    {
      "O": "autogenerated",
      "OU": "etcd cluster",
      "L": "the internet"
    }
  ]
}

Now generate the certs

make

Start etcd

Change directories back to the root of the project and run the following command to start etcd:

./bin/etcd --listen-client-urls=https://127.0.0.1:2379 --advertise-client-urls=https://127.0.0.1:2379--client-cert-auth=true --cert-file=hack/tls-setup/certs/etcd1.pem --key-file=hack/tls-setup/certs/etcd1-key.pem --trusted-ca-file=hack/tls-setup/certs/ca.pem

Attempt to Connect via V3 API

From a separate terminal window, attempt to connect via the V3 API using the following command:

ETCDCTL_API=3 ./bin/etcdctl --endpoints=https://127.0.0.1:2379 --cert=hack/tls-setup/certs/etcd2.pem --key=hack/tls-setup/certs/etcd2-key.pem --cacert=hack/tls-setup/certs/ca.pem member list

This will return the rather unhelpful:

Error: context deadline exceeded

Attempt to Connect via V2 API

Now attempt to connect via the V2 API using the following command:

ETCDCTL_API=2 ./bin/etcdctl --endpoints=https://127.0.0.1:2379 --cert-file=hack/tls-setup/certs/etcd2.pem --key-file=hack/tls-setup/certs/etcd2-key.pem --ca-file=hack/tls-setup/certs/ca.pem member list

This will return the much more helpful:

client: etcd cluster is unavailable or misconfigured; error #0: x509: certificate is valid for 3.8.121.201, 46.4.19.20, not 127.0.0.1

Related

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 4
  • Comments: 16 (11 by maintainers)

Most upvoted comments

Thanks @dahc and @wenjiaswe for taking a look at this. I’ve added steps to reproduce the issue at the top.

@xiang90 Not yet.

@spzala Do you have bandwidth to help bump grpc version? The fix we need is included in 1.21 and above. Otherwise I can do it sometime next week (I’m currently out of country on vacation).