hierarchical-namespaces: HNC Controller is reporting "Operation cannot be fulfilled on hncconfigurations.hnc.x-k8s.io "config": the object has been modified"
HNC Controller throws periodic errors:
Operation cannot be fulfilled on hncconfigurations.hnc.x-k8s.io "config": the object has been modified; please apply your changes to the latest version and try again
{"level":"error","ts":1630010586.251877,"logger":"reconcilers.HNCConfiguration","msg":"Couldn't write singleton","error":"Operation cannot be fulfilled on hncconfigurations.hnc.x-k8s.io \"config\": the object has been modified; please apply your changes to the latest version and try again"}
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 23 (12 by maintainers)
It can be ignored - that’s the great thing about these controllers, they’re pretty robust to failures. With that said, I’d kind of like to understand why it’s happening in the first place. Learning that Argo was occasionally writing the HNCConfiguration object would certainly be an explanation. But it’s not serious and can be ignored as long as it only happens from time to time.
lmk if you’d like to add an
--unpropagated-labels
option, I’d certainly be in favour of adding it.Yep - this is exactly what we did to stop that issue, @minchao. 😃
That’s odd. I must have been thinking of something else, we prevented deletions in v0.6 and I don’t see any changes to that file that would have disabled it. I’m not sure what you saw in v0.7, and that doesn’t show up in the list of issues fixed in v0.8.
Will
--unpropagated-annotations
work for you?Back to the original issue - how often are you seeing that error in the log? I looked over the code and we only ever write to it from a single goroutine (I’m about 80% sure) so there shouldn’t be a concurrency problem. Any chance Argo is changing things behind the scenes?
The error should be harmless as HNC will just try again from scratch if something goes wrong, but it’s a bit weird.