kubernetes: [Flaking Test] Conformance-GCE-master-kubetest2 (client rate limiter Wait returned an error)

Failure cluster c538bc0857d4164bb14a

Error text:

[FAILED] client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline
In [It] at: test/e2e/apimachinery/resource_quota.go:1191 @ 11/09/23 14:53:30.003

TestGrid: https://testgrid.k8s.io/sig-release-master-blocking#Conformance - GCE - master - kubetest2

Recent failures:

11/15/2023, 5:50:05 PM ci-kubernetes-e2e-gci-gce 11/15/2023, 1:48:47 PM ci-kubernetes-gce-conformance-latest-kubetest2 11/15/2023, 10:47:10 AM ci-kubernetes-gce-conformance-latest-kubetest2 11/15/2023, 4:20:04 AM ci-kubernetes-e2e-ubuntu-gce-network-policies 11/15/2023, 3:54:04 AM ci-cos-containerd-e2e-cos-gce

/kind flake

/sig api-machinery

About this issue

Original URL
State: closed
Created 8 months ago
Comments: 25 (25 by maintainers)

Most upvoted comments

ok, my bad, I didn’t follow the code right, the resync period for the resource quota controller is an option in

https://github.com/kubernetes/kubernetes/blob/94f15bbbcbe952762b7f5e6e3f77d86ecec7d7c2/cmd/kube-controller-manager/app/core.go#L552

and the default is 5 minutes

https://github.com/kubernetes/kubernetes/blob/94f15bbbcbe952762b7f5e6e3f77d86ecec7d7c2/pkg/controller/resourcequota/config/v1alpha1/defaults.go#L40-L42

that matches with the observation on the failed test , everything adds up now

aojea on Nov 18, 2023

@aojea: Using assertions inside a wait.Poll callback as in https://github.com/kubernetes/kubernetes/blob/8254569d50d6bbe8d60bcdfb16eac337f6e60514/test/e2e/apimachinery/resource_quota.go#L1189-L1191 is wrong. An error should be returned instead. Or better yet, don’t use wait.Poll…

An assertion inside a gomega.Eventually is not ideal either, but I think gomega.Eventually would handle it better than wait.Poll .

pohly on Nov 16, 2023