etcd: clientv3 doesn't renew the auth token

The etcd clientv3 doesn’t renew the auth token when it expires, it just fails with invalid auth token. It happens with both simple and jwt token types.

Steps to reproduce:

  1. Start etcd with etcd --auth-token simple --auth-token-ttl 5
  2. Enable authentication
etcdctl user add root
etcdctl user  grant-role  root root
etcdctl auth enable
# Verify that it works
etcdctl --user root:root put test test
etcdctl --user root:root get test
  1. Run the following Go program
package main

import (
	"context"
	"fmt"
	"log"
	"time"

	clientv3 "go.etcd.io/etcd/client/v3"
	"google.golang.org/grpc"
)

func main() {
	client, err := clientv3.New(clientv3.Config{
		Context:   context.Background(),
		Endpoints: []string{"127.0.0.1:2379"},
		Username:  "root",
		Password:  "root",
		DialOptions: []grpc.DialOption{
			grpc.WithBlock(),
		},
	})
	if err != nil {
		log.Fatal(err)
	}

	resp, err := client.Get(context.Background(), "test")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(resp.Kvs)

	time.Sleep(6 * time.Second)

	_, err = client.Get(context.Background(), "test")
	if err != nil {
		log.Fatal(err)
	}
}

Output:

[key:"test" create_revision:2 mod_revision:2 version:1 value:"test" ]
{"level":"warn","ts":"2021-10-11T11:38:59.474+0200","logger":"etcd-client","caller":"v3@v3.5.0/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0001ef180/#initially=[127.0.0.1:2379]","attempt":0,"error":"rpc error: code = Unauthenticated desc = etcdserver: invalid auth token"}

As you can see, the first client.Get succeeded in fetching the test key, but the second one fails with invalid auth token.

Am I missing something simple here?

EDIT: If you make 10 requests while sleeping for 1 second in-between, the error does not show up.

	for i := 1; i <= 10; i++ {
		time.Sleep(1 * time.Second)
                // no error
		resp, err := client.Get(context.Background(), "test")
		if err != nil {
			log.Fatal(err)
		}

		fmt.Println(resp.Kvs)
	}

	time.Sleep(6 * time.Second)
        // return invalid auth token error
	_, err = client.Get(context.Background(), "test")
	if err != nil {
		log.Fatal(err)
	}

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (11 by maintainers)

Most upvoted comments

thanks for checking @ahrtr , I agree with your findings. The warning log message is not harmful at all. Also can we compile the example program if we have the cloned repository of etcd in GOPATH, right? @ljupchokotev @cfz could you check that it’s still reproducible on main? I’m also backporting the fix to release-3.5: #13477

hi @mitake,

i checked latest main branch, and your backport branch (https://github.com/mitake/etcd/tree/backport-13308-to-3.5), both are failed with my test case described in #12385. maybe you can help me to double check? (the original test case is using v3.4 module.)

@cfz The following is my go.mod file for the example, and it’s working. Please have a try and let me know the result.

module etcd/tokenRenewDemo

go 1.17

require (
	go.etcd.io/etcd/client/v3 v3.5.1
	google.golang.org/grpc v1.41.0
)

require (
	github.com/coreos/go-semver v0.3.0 // indirect
	github.com/coreos/go-systemd/v22 v22.3.2 // indirect
	github.com/gogo/protobuf v1.3.2 // indirect
	github.com/golang/protobuf v1.5.2 // indirect
	go.etcd.io/etcd/api/v3 v3.5.1 // indirect
	go.etcd.io/etcd/client/pkg/v3 v3.5.1 // indirect
	go.uber.org/atomic v1.7.0 // indirect
	go.uber.org/multierr v1.6.0 // indirect
	go.uber.org/zap v1.17.0 // indirect
	golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4 // indirect
	golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40 // indirect
	golang.org/x/text v0.3.5 // indirect
	google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c // indirect
	google.golang.org/protobuf v1.26.0 // indirect
)