crossplane: ProviderConfigUsage does not work when deletion policy is set to Foreground
What happened?
While testing my function tee prototype with a claim using compositeDeletePolicy: Foreground, I noticed that ProviderConfigUsage does not work as expected.
So, my function outputs a provider kubernetes ProviderConfig resource (with InjectedIdentity) and an Object wrapping a ConfigMap and using that ProviderConfig. When I delete my claim, I noticed that the Object resource is orphaned since the ProviderConfig was deleted before the Object itself. I verified that the ProviderConfig resource had the in-use.crossplane.io finalizer and the corresponding ProviderConfigUsage resource existed.
After some investigation and thought, I believe this is a bug with the current implementation of ProviderConfigUsage when it is used with deletion policy Foreground. The issue is, with this mode of deletion propagation, resources are being deleted starting from the bottom of the tree and hence ProviderConfigUsage being owned by the Object is deleted before the actual Object is deleted. This removes the blockage on ProviderConfig and it got deleted without waiting the deletion of the Object.
@bobh66 I am wondering if you had encountered something like this before?
How can we reproduce it?
kubectl apply -f setup.yaml
kubectl apply -f claim.yaml
# wait until claim is ready
kubectl delete -f claim.yaml
setup.yaml
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-nop
spec:
package: xpkg.upbound.io/crossplane-contrib/provider-nop:v0.2.0
ignoreCrossplaneConstraints: true
---
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
name: function-dummy
spec:
# NOTE(negz): This is currently manually pushed. See README.md at
# https://github.com/crossplane-contrib/function-dummy.
package: xpkg.upbound.io/crossplane-contrib/function-dummy:v0.2.0
---
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
name: function-tee
spec:
package: turkenh/function-tee:v0.1.0
---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xnopresources.nop.example.org
spec:
group: nop.example.org
names:
kind: XNopResource
plural: xnopresources
claimNames:
kind: NopResource
plural: nopresources
connectionSecretKeys:
- test
versions:
- name: v1alpha1
served: true
referenceable: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
coolField:
type: string
required:
- coolField
status:
type: object
properties:
coolerField:
type: string
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: xnopresources.nop.example.org
spec:
compositeTypeRef:
apiVersion: nop.example.org/v1alpha1
kind: XNopResource
mode: Pipeline
pipeline:
- step: be-a-dummy
functionRef:
name: function-dummy
input:
apiVersion: dummy.fn.crossplane.io/v1beta1
kind: Response
# This is a YAML-serialized RunFunctionResponse. function-dummy will
# overlay the desired state on any that was passed into it.
response:
desired:
composite:
resource:
status:
coolerField: "I'M COOLER!"
resources:
nop-resource-1:
resource:
apiVersion: nop.crossplane.io/v1alpha1
kind: NopResource
spec:
forProvider:
conditionAfter:
- conditionType: Ready
conditionStatus: "False"
time: 0s
- conditionType: Ready
conditionStatus: "True"
time: 10s
ready: READY_TRUE
results:
- severity: SEVERITY_NORMAL
message: "I am doing a compose!"
- step: tee
functionRef:
name: function-tee
claim.yaml
apiVersion: nop.example.org/v1alpha1
kind: NopResource
metadata:
namespace: default
name: apiextensions-composition-functions
spec:
coolField: "I'm cool!"
# This is necessary to ensure the claim's MRs are actually gone before we
# delete the Provider - https://github.com/crossplane/crossplane/issues/4251
compositeDeletePolicy: Foreground
What environment did it happen in?
Crossplane version: v1.14.0-rc.0.369.g293439e0
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 8
- Comments: 22 (14 by maintainers)
Since the
ProviderConfigUsageis owned by the Managed Resource, and does not have a finalizer, it is deleted immediately when the MR is marked deleted. This opens up a window where the MR exists but theProviderConfighas no knowledge of it so theProviderConfigcould be deleted while the MR is still trying to delete the external resource.I think it would be better to put a finalizer on the
ProviderConfigUsageand have crossplane-runtime remove the finalizer after a successfull call to theDeletemethod./fresh
/fresh
/fresh
/fresh