pulumi-kubernetes: `kubernetes.yaml.*` produces a `TypeError: Cannot read properties of undefined`

What happened?

If the default kubernetes provider is disabled, the kubernetes.yaml “submodule” doesn’t work as expected. I’m not entirely sure if this is related to disabling the default provider, but I assume it is.

Steps to reproduce

I’ve created a small reproduction in omninonsense/pulumi-repro.

But the general steps to reproduce it should probably be:

Disable the default kubernetes provider (dunno if this is a red herring)
Try to use kubernetes.yaml.ConfigFile (or others) with an explicit opts.provider

Expected Behavior

I expected it to apply the manifests using the provider passed via options.

Actual Behavior

I received the following error:

Diagnostics:
pulumi:pulumi:Stack (pulumi-repro-dev):
	error: TypeError: Cannot read properties of undefined (reading 'map')
			at /Users/nino/Documents/silicoai/pulumi-repro/node_modules/@pulumi/yaml/yaml.ts:2993:14
			at processTicksAndRejections (node:internal/process/task_queues:95:5)

Versions used

pulumi about
CLI
Version      3.34.1
Go Version   go1.18.3
Go Compiler  gc

Plugins
NAME        VERSION
aws         5.8.0
command     0.3.0
docker      3.2.0
eks         0.40.0
kubernetes  3.19.3
nodejs      unknown

Host
OS       darwin
Version  12.4
Arch     arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v18.3.0'

Backend
Name           pulumi.com
URL            https://app.pulumi.com/omninonsense
User           omninonsense
Organizations  omninonsense, EpicGames, silico

Dependencies:
NAME                VERSION
@pulumi/aws         5.8.0
@pulumi/awsx        0.40.0
@pulumi/eks         0.40.0
@pulumi/kubernetes  3.19.3
@pulumi/pulumi      3.34.1
@types/node         14.18.21

Pulumi locates its logs in /var/folders/xm/qbl84xzs55g88fvzdn63nkdr0000gn/T/ by default
warning: Failed to get information about the current stack: No current stack

Additional context

There’s a hardcoded AWS providerCredentialOpts.profileName (set to "sandbox"), which will probably need editing in the repro repo. There’s also a bunch of AWS SSO and RBAC stuff, but I think that can largely be ignored (I yanked this from some existing code).

Also, the issue isn’t present when “manually” creating k8s resources using the native TypeScript API.

Also, I realise the title is a bit poor, but I’m not sure

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you’ve opened one already).

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 14
Comments: 21 (12 by maintainers)

Most upvoted comments

I think I have the same issue in my small demo repo now. I create a aks cluster, than I want to use the config when creating the cluster to create a provider that I use to apply some kubernetes manifests, and it fails with:

TypeError: Cannot read property ‘map’ of undefined

If I use the default provider it seems to work.

@Frassle , I don’t think allowing immediate values to create a provider is the way to go, especially with kubernetes. I can definitely see scenarios where you have your core infrastructure where you create your cluster AND also add common things to the cluster like istio, linkerd, traefik and things like that. In those scenarios I do think you want to create the cluster and then use the config you get from the cluster creation to create the provider to apply those things.

mastoj on Jul 14, 2022

Confirmed this is happening without disabling default kubernetes provider - you just need to use ConfigFile on the non-default provider. The problem seems to be that the invoke to kubernetes:yaml:decode returns an empty response. Interestingly I am not seeing an actual RPC call being made to the provider here but I need to do some more investigation here to confirm. Marked P1 and assigned myself.

viveklak on Jun 28, 2022

A hacky workaround for now would be to either:

Run the pulumi up with --skip-preview
Move the ConfigFile resource into an apply block wrapping the unknown configuration options of the provider, e.g.: if k8sprovider relies onkubeconfig which is unknown (output of cluster creation), move the ConfigFile to:

provider.kubeconfig.apply(_ => new k8s.yaml.ConfigFile("confFile"...))

viveklak on Jun 29, 2022

Ran into the same issue with .NET SDK. It manifests somewhat differently by throwing NullReferenceException when creating ImmutableArray result. The issue is annoying since it is not possible to use previews when the stack relies on the newly created cluster credentials.

The workaround we have tried is to conditionally set provider arguments depending on whether the code runs in preview or not, like so:

new Kubernetes.Provider ("kube",
  Deployment.Instance.IsDryRun
    ? new Kubernetes.ProviderArgs ()
    : new Kubernetes.ProviderArgs { KubeConfig = cluster.KubeConfig })

What are the possible downsides of this approach?

For the record, the change that triggers this new behavior is 763ddcadd7843ea94e87239383da8d0a436b92bc

illinar on Jul 28, 2022

@omninonsense ah so this is failing with an empty/brand new stack?

viveklak on Jun 27, 2022