terraform-provider-kubectl: kubectl_manifest returning "Provider produced inconsistent result after apply" in Kubernetes 1.20

Description

I am not able to apply kubectl_manifest to Kubernetes 1.20, I receive an error. Although I can confirm the resources are created, the plan will exit with an error breaking the execution. Applying the same configuration in Kubernetes 1.18/1.19 works without returning any errors.

Steps to reproduce

Whilst trying to apply the following

resource "helm_release" "cert_manager" {
  name       = "cert-manager"
  repository = "https://charts.jetstack.io"
  chart      = "cert-manager"

  namespace  = "cert-manager"
  create_namespace = true

  set {
    name  = "installCRDs"
    value = "true"
  }
}

resource "kubernetes_secret" "cloudflare_token_api" {
  metadata {
    name = "cloudflare-token"
    namespace = "cert-manager"
  }

  data = {
    api-token = "secret_token"
  }

  depends_on = [
    helm_release.cert_manager
  ]
}

resource "kubectl_manifest" "issuer" {
  yaml_body = <<YAML
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: test
spec:
  acme:
    email: your@email.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: cloudflare-acme-private-key
    solvers:
    - dns01:
        cloudflare:
          email: your@email.com
          apiTokenSecretRef:
            name: cloudflare-token
            key: api-token
YAML

  depends_on = [
    helm_release.cert_manager,
    kubernetes_secret.cloudflare_token_api
  ]
}

I receive the error Root resource was present, but now absent.

module.cert-manager.helm_release.cert_manager: Creating...
module.cert-manager.helm_release.cert_manager: Still creating... [10s elapsed]
module.cert-manager.helm_release.cert_manager: Still creating... [20s elapsed]
module.cert-manager.helm_release.cert_manager: Creation complete after 27s [id=cert-manager]
module.cert-manager.kubernetes_secret.cloudflare_token_api: Creating...
module.cert-manager.kubernetes_secret.cloudflare_token_api: Creation complete after 1s [id=cert-manager/cloudflare-api-token]
module.cert-manager.kubectl_manifest.issuer: Creating...

Error: Provider produced inconsistent result after apply

When applying changes to module.cert-manager.kubectl_manifest.issuer, provider
"registry.terraform.io/gavinbunney/kubectl" produced an unexpected new value:
Root resource was present, but now absent.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

This is only occurring for me on Kubernetes 1.20, I have tested in 1.18/1.19 and it works

module.cert-manager.helm_release.cert_manager: Creating...
module.cert-manager.helm_release.cert_manager: Still creating... [10s elapsed]
module.cert-manager.helm_release.cert_manager: Still creating... [20s elapsed]
module.cert-manager.helm_release.cert_manager: Still creating... [30s elapsed]
module.cert-manager.helm_release.cert_manager: Still creating... [40s elapsed]
module.cert-manager.helm_release.cert_manager: Still creating... [50s elapsed]
module.cert-manager.helm_release.cert_manager: Creation complete after 57s [id=cert-manager]
module.cert-manager.kubernetes_secret.cloudflare_token_api: Creating...
module.cert-manager.kubernetes_secret.cloudflare_token_api: Creation complete after 1s [id=cert-manager/cloudflare-api-token]
module.cert-manager.kubectl_manifest.issuer: Creating...
module.cert-manager.kubectl_manifest.issuer: Creation complete after 2s [id=/apis/cert-manager.io/v1/namespaces/default/issuers/test]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 11
  • Comments: 15 (6 by maintainers)

Commits related to this issue

Most upvoted comments

Released as v1.10

I think that the issue is that selfLink propagation is disabled in K8S 1.20 and will be removed in 1.21.

I ran TF_LOG=trace terraform apply and saw this line (timestamps removed for clarity:

[DEBUG] plugin.terraform-provider-kubectl_v1.9.4: [INFO] myns/myresource perform apply of manifest
[DEBUG] plugin.terraform-provider-kubectl_v1.9.4: [INFO] myns/myresource manifest applied, fetch resource from kubernetes
[DEBUG] plugin.terraform-provider-kubectl_v1.9.4: [DEBUG] myns/myresource fetched successfully, set id to: 
[DEBUG] plugin.terraform-provider-kubectl_v1.9.4: [DEBUG] myns/myresource fetch from kubernetes
[DEBUG] kubectl_manifest.test[0]: apply errored, but we're indicating that via the Error pointer rather than returning it: Provider produced inconsistent result after apply: When applying changes to kubectl_manifest.test[0], provider "registry.terraform.io/gavinbunney/kubectl" produced an unexpected new value: Root resource was present, but now absent.

This is a bug in the provider, which should be reported in the provider's own issue tracker.

See the line fetched successfully, set id to: … That corresponds to this source line:

d.SetId(response.GetSelfLink())
log.Printf("[DEBUG] %v fetched successfully, set id to: %v", manifest, d.Id())

So this provider is tracking resources with selfLinks, but K8S 1.20 doesn’t have them anymore. These IDs need to be extracted / generated some other way going forward.