pulumi: Can't disable default Kubernetes provider in 3.23.0 for yaml.ConfigFile

Hello!

• Vote on this issue by adding a 👍 reaction
• To contribute a fix for this issue, leave a comment (and link to your pull request, if you’ve opened one already)

Issue details

I am trying out the new Pulumi 3.23.0 capability of disabling the default providers, and have this in my stack:

  pulumi:disable-default-providers:
  - aws
  - kubernetes

This seems to work well for AWS, but I am having an odd issue with the Kubernetes one. Specifically, I have the following resource:

    _, err = yaml.NewConfigFile(ctx, "certmanager-deploy-file", &yaml.ConfigFileArgs{
        File: "./cert-manager.yaml",
        Transformations: []yaml.Transformation{
            // We need to make two modifications:
            // 1. Add the role ARN for IRSA
            // 2. Set the fsGroup for IRSA token mapping
            // Docs here: https://cert-manager.io/docs/configuration/acme/dns01/route53/#eks-iam-role-for-service-accounts-irsa
            func(state map[string]interface{}, opts ...pulumi.ResourceOption) {
                metadata := state["metadata"].(map[string]interface{})
                name := metadata["name"]
                if state["kind"] == "ServiceAccount" && name == "cert-manager" {
                    var annotations map[string]interface{}
                    if v, ok := metadata["annotations"]; !ok {
                        annotations = make(map[string]interface{})
                        metadata["annotations"] = annotations
                    } else {
                        annotations = v.(map[string]interface{})
                    }
                    annotations["eks.amazonaws.com/role-arn"] = irsaRole.Arn
                }
                if state["kind"] == "Deployment" && name == "cert-manager" {
                    deploymentSpec := state["spec"].(map[string]interface{})
                    template := deploymentSpec["template"].(map[string]interface{})
                    podSpec := template["spec"].(map[string]interface{})
                    podSpec["securityContext"] = map[string]interface{}{
                        "fsGroup": 1001,
                    }
                }

            },
        },
    }, pulumi.DependsOn([]pulumi.Resource{irsaRole}), pulumi.Provider(eksConfig.Provider))
    if err != nil {
        return nil, err
    }

Where eksConfig.Provider is constructed as the result of an eks.Cluster creation:

    k8sProvider, err := providers.NewProvider(ctx, "k8s-ssa-provider", &providers.ProviderArgs{
        Kubeconfig: kubeconfig,
    })
    if err != nil {
        return nil, err
    }

When I run this with the default Kubernetes one disabled, I get this error:

    error: program failed: 1 error occurred:
        * decoding YAML: rpc error: code = Unknown desc = unknown provider ''
    exit status 1

There is not any more info in the logs even if I set logging to 9. If I enable the Kubernetes default provider, it works just fine, even though I am passing an explicit provider here. I know it is this resource that’s causing the issue as if I comment it out, things work.

Looking at the state file for this resource, I notice:

{
    "urn": "urn:pulumi:prod-us-west-1::okera-infra-regions::kubernetes:yaml:ConfigFile::certmanager-deploy-file",
    "custom": false,
    "type": "kubernetes:yaml:ConfigFile",
    "parent": "urn:pulumi:prod-us-west-1::okera-infra-regions::pulumi:pulumi:Stack::okera-infra-regions-prod-us-west-1",
    "dependencies": [
        "urn:pulumi:prod-us-west-1::okera-infra-regions::aws:iam/role:Role::cert-manager-iam-role"
    ]
},

So it seems like this resource does not have a provider set. Maybe because it’s a ComponentResource? I am not sure.

Also note that I’ve always passed my custom provider here, even prior to disabling the default Kubernetes one, so there is no explicit provider change.

Is this a bug or am I doing something unexpected here?

Steps to reproduce

See above.

Expected: I can disable the Kubernetes default provider and everything will work if I pass a default provider. Actual: I can’t disable the Kubernetes default provider.