crossplane: Crossplane fails to synchronize claims with XRs
What happened?
From time to time I see that claims and XRs loose sync such as here (see second resource):
❯ kubectl get xirsas
NAME API SYNCED READY COMPOSITION AGE
acm-pca-issuer-nw-eu-west-3-main infra.nexthink.com/v1alpha3 True True custom.policy.xirsas.infra.nexthink.com 9m30s
acm-pca-issuer-nw-us-east-2-main infra.nexthink.com/v1alpha3 custom.policy.xirsas.infra.nexthink.com 9m31s
collector-traffic-nw-eu-west-3-main infra.nexthink.com/v1alpha3 True True custom.auth.xirsas.infra.nexthink.com 22h
[...]
And this does not change until I do a rollout restart of crossplane deployment. Logs in crossplane deployment look to be on a loop continuously prompting logs such these:
crossplane-7c898b5fdf-rf7mg universal-crossplane {"level":"info","ts":"2024-02-16T10:15:46Z","logger":"crossplane","msg":"Enqueueing composite resource because managed resource changed","controller":"defined/compositeresourcedefinition.apiextensions.crossplane.io","request":{"name":"xcertmanagers.infra.nexthink.com"},"uid":"c3eb8136-bcc4-4c98-8832-d782e62cb2b2","version":"168408426","name":"xcertmanagers.infra.nexthink.com","name":"cert-manager-nw-eu-west-3-main","mrGVK":"infra.nexthink.com/v1alpha1, Kind=XAcmPcaIssuer","mrName":"pca-plugin-nw-eu-west-3-main"}
crossplane-7c898b5fdf-rf7mg universal-crossplane {"level":"info","ts":"2024-02-16T10:15:46Z","logger":"crossplane","msg":"Enqueueing composite resource because managed resource changed","controller":"defined/compositeresourcedefinition.apiextensions.crossplane.io","request":{"name":"xcertmanagers.infra.nexthink.com"},"uid":"c3eb8136-bcc4-4c98-8832-d782e62cb2b2","version":"168408426","name":"xcertmanagers.infra.nexthink.com","name":"cert-manager-nw-eu-west-3-main","mrGVK":"infra.nexthink.com/v1alpha1, Kind=XAcmPcaIssuer","mrName":"pca-plugin-nw-eu-west-3-main"}
crossplane-7c898b5fdf-rf7mg universal-crossplane {"level":"info","ts":"2024-02-16T10:15:46Z","logger":"crossplane","msg":"Enqueueing composite resource because managed resource changed","controller":"defined/compositeresourcedefinition.apiextensions.crossplane.io","request":{"name":"xcertmanagers.infra.nexthink.com"},"uid":"c3eb8136-bcc4-4c98-8832-d782e62cb2b2","version":"168408426","name":"xcertmanagers.infra.nexthink.com","name":"cert-manager-nw-eu-west-3-main","mrGVK":"infra.nexthink.com/v1alpha1, Kind=XAcmPcaIssuer","mrName":"pca-plugin-nw-eu-west-3-main"}
crossplane-7c898b5fdf-rf7mg universal-crossplane {"level":"info","ts":"2024-02-16T10:15:46Z","logger":"crossplane","msg":"Enqueueing composite resource because managed resource changed","controller":"defined/compositeresourcedefinition.apiextensions.crossplane.io","request":{"name":"xcertmanagers.infra.nexthink.com"},"uid":"c3eb8136-bcc4-4c98-8832-d782e62cb2b2","version":"168408426","name":"xcertmanagers.infra.nexthink.com","name":"cert-manager-nw-eu-west-3-main","mrGVK":"infra.nexthink.com/v1alpha1, Kind=XAcmPcaIssuer","mrName":"pca-plugin-nw-eu-west-3-main"}
How can we reproduce it?
What environment did it happen in?
Crossplane version: universal-crossplane-1.14.5-up.1
EKS v1.27.9-eks-5e0fdde
Relevant PRs
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 19 (18 by maintainers)
@negz yes, I can reproduce it:
add
- --enable-realtime-compositionsto crossplane deployment argsInstall configuration-aws-network from the marketplace: https://marketplace.upbound.io/configurations/upbound/configuration-aws-network/v0.24.0
Apply a sample XR: https://github.com/upbound/configuration-aws-network/blob/main/examples/xnetworks/configuration-aws-network.yaml:
The composite will not have any events on it:
/fresh