terraform-provider-kubernetes: K8s upgrade to 1.22.1 - Failed to create Ingress 'XXX' because: the server could not find the requested resource (post ingresses.extensions)

Terraform Version, Provider Version and Kubernetes Version

Terraform version: 1.0.5
Kubernetes provider version: 2.4.1
Kubernetes version: 1.22.1

Affected Resource(s)

  • kubernetes-ingress

Terraform Configuration Files

# main.tf
resource "kubernetes_ingress" "example_ingress" {
  metadata {
    name = "example-ingress"
  }

  spec {
    backend {
      service_name = "MyApp1"
      service_port = 8080
    }

    rule {
      http {
        path {
          backend {
            service_name = "MyApp1"
            service_port = 8080
          }

          path = "/app1/*"
        }

        path {
          backend {
            service_name = "MyApp2"
            service_port = 8080
          }

          path = "/app2/*"
        }
      }
    }

    tls {
      secret_name = "tls-secret"
    }
  }
}
# versions.tf
terraform {
  required_providers {
    kubernetes = {
      source = "hashicorp/kubernetes"
      version = ">= 2.4.1"
    }
  }
  required_version = ">= 1.0.5"
}
# provider.tf
provider "kubernetes" {
  host = XXX
  client_certificate = XXX
  client_key = XXX
  cluster_ca_certificate = XXX
}

Panic Output

kubernetes_ingress.example_ingress: Creating...
╷
│ Error: Failed to create Ingress 'default/example-ingress' because: the server could not find the requested resource (post ingresses.extensions)
│
│   with kubernetes_ingress.example_ingress,
│   on main.tf line 1, in resource "kubernetes_ingress" "example_ingress":
│    1: resource "kubernetes_ingress" "example_ingress" {

Steps to Reproduce

  1. terraform apply

Expected Behavior

The run should have completed succesfully.

Actual Behavior

After upgrading k8s to v1.22.1 and running a plan with a kubernetes_ingress controller, the ingress could not be created because the server could not find the requested resource (post ingresses.extensions)

Important Factoids

We are using Exoscale SKS as kubernetes service and terraform.io cloud. We have tested the above configuration locally as well.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 30
  • Comments: 18 (2 by maintainers)

Most upvoted comments

“The extensions/v1beta1 and networking.k8s.io/v1beta1 API versions of Ingress is no longer served as of v1.22.”: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122

please update the provider:

https://github.com/hashicorp/terraform-provider-kubernetes/blob/98ff5e2ead2970418d1e0f1c50eb8e5307d576ad/kubernetes/resource_kubernetes_ingress.go#L177

Thank you, kind regards

Friendly question: When is a release expected regarding this fix?

The answer to my question: days! Version 2.7.0 of the provider supports 1.22 for ingress.

For me, I had to make two changes for my simple ingress without any rules.

First, change to using the new v1:

resource "kubernetes_ingress" "frontend" {
resource "kubernetes_ingress_v1" "frontend" {

Second, update how the spec was formed:

  spec {
    backend {
      service_name = kubernetes_service.frontend.metadata.0.name
      service_port = kubernetes_service.frontend.spec.0.port.0.port
    }
  }
 spec {
    default_backend {
      service {
        name = kubernetes_service.frontend.metadata.0.name
        port {
          number = kubernetes_service.frontend.spec.0.port.0.port
        }
      }
    }
  }

Yes, version 2.7.0 has finally been released as a tag (no github release yet, also available here https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs)

Updating to kubernetes_ingress_v1 works good in a green field environment. Updating an already existing ingress can be a struggle, as terraform has trouble creating&deleting an ingress with the same name, even though it notices the difference between kubernetes_ingress vs kubernetes_ingress_v1.

We ended up removing the existing beta ingresses, after that another run created the new ingresses just fine.

@nickreynke while we wait for a new release, we have built a custom release and use this in terraform cloud as described here: https://www.terraform.io/docs/cloud/run/install-software.html#in-house-providers

Hi @hashicorp, any news or update regarding this topic?

We haven’t noticed any changes in the main branch yet:

https://github.com/hashicorp/terraform-provider-kubernetes/blob/28e968e0fc08e39d5a3af50015a4df7c4c58de83/kubernetes/resource_kubernetes_ingress.go#L177

Thank you, kind regards

@eake I’m curious how well creating a new ingress works, since pathType is required to be set and I did not see anything related to it. Besides now not allowing named ports to be used (port.name). Have you ran the full test suite?

This is still a problem on version 2.7.1, using latest microk8s

$ snap list microk8s
Name      Version  Rev   Tracking     Publisher   Notes
microk8s  v1.22.4  2695  1.22/stable  canonical✓  classic

~From a fresh install of microk8s, enable ingress and dns addon and try to apply a terraform file like the original example, but ensure that kubernetes provider is upgraded to 2.7.1. It fails with the same error.~

I misunderstood the previous comment, the actual resource name used should now be kubernetes_ingress_v1 instead of kubernetes_ingress

@vgn-tl Thanks placing custom build binary at ~/.terraform.d/pluginsregistry.terraform.io/hashicorp/kubernetes/2.6.2/linux_amd64/terraform-provider-kubernetes and replacing

terraform {
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "2.6.2"
    }
  }
}

resource "kubernetes_ingress" "name { with resource "kubernetes_ingress_v1" "name" { supports v1.22 very well.

in-case someone wants custom binary, download it from https://file.io/kWDZ7rdLWyoD

git sha commit 92a70886b3487b7d39a56a6bbd87136a016e2303 (HEAD -> main, origin/main , origin/HEAD)