apisix-ingress-controller: bug: Special circumstance may cause expired configuration be PUT to APISIX

Issue description

The code show that synced failure Event will be requeue to the workQueue. But the Event may be expired while resync

Environment

your apisix-ingress-controller version (output of apisix-ingress-controller version --long):
your Kubernetes cluster version (output of kubectl version):
if you run apisix-ingress-controller in Bare-metal environment, also show your OS version (uname -a):

Minimal test code / Steps to reproduce

to reproduce this bug by adding two Endpoints to one Deployment

Adding Endpoint Event1 arrived
Adding Endpoint Event2 arrived
The workQueue has two events
The Event1 sync failure, duo to the network error. and The even1 be requeue
The Event2 sync successfully. There are two Nodes in the APISIX Upstream
The Event1 sync successfully. There is only one Node in the APISIX Upstream

Actual result

only one Node in the APISIX Upstream

Error log

no error

Expected result

two Nodes in the APISIX Upstream

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 16 (16 by maintainers)

Most upvoted comments

We can just can change the objects entering the queue, just use the namespace/name is enough (instead of the pointer to the endpoint object), just like other controllers.

This is a good way to write a controller, see https://github.com/kubernetes/kubernetes/blob/da9a46bac2f0b389405efad28b165543d94a3f73/pkg/controller/endpoint/endpoints_controller.go#L378-L404.

tokers on Dec 21, 2021

why using == here when compare resourceVersion? It should be >=.

Should we submit a PR to fix this first?

nayihz on Mar 6, 2022

After checking the code in endpointController, here is strange ，why using == here when compare resourceVersion? It should be >=.

gxthrj on Mar 6, 2022

If I am correct, we don’t save the updated object when we get the update event, instead, we save the key and fetch the object when we handle it, so even we handle an old event, we still get the newest object and we can compare the ResourceVersion so that stale update can be avoid.

tokers on Mar 6, 2022