argo-cd: Resources with tracking method annotation incorrectly marked as out-of-sync/requires pruning
Checklist:
- I’ve searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
- I’ve included steps to reproduce the bug.
- I’ve pasted the output of
argocd version.
Describe the bug
One of the goals of using annotations for resource tracking is, that the resource should only be considered managed if the values of the annotation match the properties of the resource. However, this does not seem to be the case.
To Reproduce
- Set
application.resourceTrackingMethodtoannotation - Create a new application, e.g.
kustomize-guestbookfromhttps://github.com/argoproj/argocd-example-apps, with a target namespace ofguestbook - Sync the application. Make sure that Argo CD has put on tracking annotations on the live resources instead of the label, e.g.
annotations:
argocd.argoproj.io/tracking-id: 'guestbook:/Service:guestbook/kustomize-guestbook-ui'
- Create a new dummy resource in namespace
guestbook, for example aConfigMap:
kubectl -n guestbook create configmap foobar
- Add annotations from another resource to dummy resource:
kubectl -n guestbook annotate configmap foobar argocd.argoproj.io/tracking-id='guestbook:/Service:guestbook/kustomize-guestbook-ui'
- Observe this resource being marked as out of sync and pruning required in the application:
Expected behavior
The resource should be ignored because the properties of the resource don’t match the values in the annotation. The resource should not be considered to be managed by Argo CD, and neither marked out of sync nor for be marked for pruning.
Screenshots
Version
Latest HEAD as of the time of issue creation:
argocd-server: v2.3.0+f9b7384
BuildDate: 2022-03-04T07:39:01Z
GitCommit: f9b7384bf4d5b0197e726ca19e40518c97fd05fd
GitTreeState: clean
GoVersion: go1.17.6
Compiler: gc
Platform: linux/amd64
Kustomize Version: v4.4.1 2021-11-11T23:36:27Z
Helm Version: v3.8.0+gd141386
Kubectl Version: v0.23.1
Jsonnet Version: v0.18.0
Logs
Paste any relevant application logs here.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 22 (19 by maintainers)
Thanks for your insight, @jessesuen.
I’m also not in favor of option 1 and think the object should be somehow displayed to the user.
The major use case for me is when an Operator creates resources and copies all labels and annotations from its Operand onto the resources it creates. So if you manage the Operand via Argo CD, and the Operator creates a ConfigMap during reconciliation, that ConfigMap will carry the same labels and annotations as the Operand, including Argo CD’s tracking labels.
Now, not all Operators create
ownerReferenceson the resources they create, and that’s where the original problem stems from. In some scenarios, the Operator even isn’t able to createownerReferenceson such resources, for example when Operand and the resource are in different namespaces, or when one of Operand or resource are cluster scoped, but the other is not.So I’m in favor of a combination of option 2 and 3 above:
If the tracking label on a resource is not self-referencing, but instead references another resource in the same application, we create a parent/child relationship between the two.
If the tracking label on a resource is not self-referencing and refers to an otherwise unknown resource (e.g. a resource not managed by current application, or a resource outside the application’s scope), we display it as top-level resource without owner.
I would also be fine to first go for the simple approach (e.g. displaying it as top-level resource) and if there’s demand, we can see how to establish parent/child relationship from the UI’s point of view.
WDYT?
As an external operator which supports whole corporate teams with lesser experience, the distinction of the sync status on a top-level resource is not so obvious for most of the teams. They wouldn’t expect a top-level resource to be non-synced resource at first look, and a small icon missing is not very clear from UX/accessibility standpoint vs. a clear parent-child relationship.
A clear pseudo parent-child relationship will not raise the “what is wrong with my deployment, where does this resource come from, what do I need to with this, how do I handle this” questions I got. While a parent-child relationship is what it is: a resource they to not care about it explicitly.
Maybe top-level non-synced resource vs. pseudo parent-child relationship can be configured as feature flag via
argocd-cmconfig map or as an container argument/environment variable on the corresponding component. This would still support the use-case (by default), where operators would expext only real ownership references as parent-child relationships and can be revisited when the feature of configuring parent-child relationship by label selectors gets implemented.Fix for this issue: #9791
@ralf-cestusio Just as a heads-up, the upcoming v2.4.9 release will contain a proper fix for the pruning bug.
The fix doesn’t seem to work in v2.4.8. Here is a repo which can reproduce the issue: https://github.com/jessesuen/bug-self-ref-pruning