pulumi: assertion failure in pulumi cli

What happened?

Trying to deploy stack with stackRef in s3 bucket.

pulumi preview 
Previewing update (gcp__us-west1__neara-latest):
     Type                 Name                                    Plan        Info
     pulumi:pulumi:Stack  pulumi-k8s-gcp__us-west1__neara-latest  running..   read pulumi:pulumi:StackReference gcp__us-west-1__k8s-anthos
panic: fatal: An assertion has failed

                                     goroutine 209 [running]:
                                                             github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
                                                                                                                                  	/home/runner/work/pulumi/pulumi/sdk/go/common/util/contract/failfast.go:23
                                                                github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.Assert(...)
                                                                                                                                   	/home/runner/work/pulumi/pulumi/sdk/go/common/util/contract/assert.go:26
                                                              github.com/pulumi/pulumi/pkg/v3/resource/deploy.NewDeleteReplacementStep(0xc000b4f290, 0xc001b4cc00, 0xc000e1d320, 0x5c)
                                    	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/step.go:316 +0x199
                                                                                                              github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*stepGenerator).generateStepsFromDiff(0xc001a8eb00, {0x7febedae7cd0, 0xc001ba7300}, {0xc0013cee40, 0x5b}, 0xc001c33c20, 0xc000d90360, 0xc001b8d380, 0x0?, 0xc000ce4f30, ...)
                                       	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/step_generator.go:995 +0x1478
                                                                                                                         github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*stepGenerator).generateSteps(0xc001a8eb00, {0x7febedae7cd0, 0xc001ba7300})
                                                                                                   	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/step_generator.go:795 +0x2618
                                       github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*stepGenerator).GenerateSteps(0xc001a8eb00, {0x7febedae7cd0?, 0xc001ba7300?})
                   	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/step_generator.go:261 +0x46
                                                                                                       github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*deploymentExecutor).handleSingleEvent(0xc001b18348, {0x2875900?, 0xc001ba7300})
                                                                                      	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/deployment_executor.go:419 +0xe5
                          github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*deploymentExecutor).Execute.func3(0xc001649980, 0xc001b18348, 0xc0016ab340, {0x2890b70, 0xc001eb8440}, {{0x38dddd0?, 0x0?, 0x0?}, 0xc0014044e0?}, {{0x0, ...}, ...}, ...)
                                                                                                   	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/deployment_executor.go:256 +0x27e
                                           github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*deploymentExecutor).Execute(0xc001b18348, {0x2890c18?, 0xc001b4c570}, {{0x7febedb4b2d0, 0xc000ada780}, 0x7fffffff, 0x0, 0x0, {{0x0, 0x0, ...}, ...}, ...}, ...)
                                                                                                          	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/deployment_executor.go:272 +0xb0a
                                                   github.com/pulumi/pulumi/pkg/v3/resource/deploy.(*Deployment).Execute(0xc000b4f290, {0x2890c18, 0xc001b4c570}, {{0x7febedb4b2d0, 0xc000ada780}, 0x7fffffff, 0x0, 0x0, {{0x0, 0x0, ...}, ...}, ...}, ...)
                                                                                                         	/home/runner/work/pulumi/pulumi/pkg/resource/deploy/deployment.go:534 +0xd0
                                         github.com/pulumi/pulumi/pkg/v3/engine.(*deployment).run.func1()
                                                                                                         	/home/runner/work/pulumi/pulumi/pkg/engine/deployment.go:289 +0x290
                                 created by github.com/pulumi/pulumi/pkg/v3/engine.(*deployment).run
                                                                                                    	/home/runner/work/pulumi/pulumi/pkg/engine/deployment.go:272 +0x35b

Steps to reproduce

Probably:

  1. Use s3 backend
  2. Deploy one stack (with GKE cluster and DB).
  3. Deploy second stack, which reference first.

Expected Behavior

Deploy stack

Actual Behavior

CLI crash.

Output of pulumi about

CLI
Version 3.46.1 Go Version go1.19.2 Go Compiler gc

Plugins NAME VERSION gcp 6.43.0 kubernetes 3.22.1 nodejs unknown

Host
OS debian Version 11.5 Arch x86_64

This project is written in nodejs: executable=‘/usr/bin/node’ version=‘v16.18.0’

Current Stack: gcp__us-west1__neara-latest

TYPE URN pulumi:pulumi:Stack urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:pulumi:Stack::pulumi-k8s-gcp__us-west1__neara-latest pulumi:providers:pulumi urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:providers:pulumi::default pulumi:providers:gcp urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:providers:gcp::default_6_43_0 pulumi:pulumi:StackReference urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:pulumi:StackReference::gcp__us-west-1__k8s-anthos gcp:compute/globalAddress:GlobalAddress urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::gcp:compute/globalAddress:GlobalAddress::ingress pulumi:providers:kubernetes urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:providers:kubernetes::gcp kubernetes:core/v1:Namespace urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:Namespace::latest kubernetes:networking.gke.io/v1:ManagedCertificate urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:networking.gke.io/v1:ManagedCertificate::managed-cert kubernetes:cloud.google.com/v1:BackendConfig urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:cloud.google.com/v1:BackendConfig::appserver-backend-config kubernetes:networking.gke.io/v1beta1:FrontendConfig urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:networking.gke.io/v1beta1:FrontendConfig::frontent-config gcp:dns/recordSet:RecordSet urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::gcp:dns/recordSet:RecordSet::ingress kubernetes:core/v1:ConfigMap urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:ConfigMap::appserver kubernetes:apps/v1:Deployment urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:apps/v1:Deployment::nginx kubernetes:core/v1:Service urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:Service::nginx kubernetes:core/v1:Service urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:Service::appserver kubernetes:apps/v1:Deployment urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:apps/v1:Deployment::appserver kubernetes:networking.k8s.io/v1:Ingress urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:networking.k8s.io/v1:Ingress::ingress pulumi:pulumi:StackReference urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:pulumi:StackReference::gcp__us-west1__k8s pulumi:providers:gcp urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:providers:gcp::default_6_42_0 pulumi:providers:kubernetes urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::pulumi:providers:kubernetes::gcp kubernetes:core/v1:Namespace urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:Namespace::latest kubernetes:networking.gke.io/v1beta1:FrontendConfig urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:networking.gke.io/v1beta1:FrontendConfig::frontent-config kubernetes:cloud.google.com/v1:BackendConfig urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:cloud.google.com/v1:BackendConfig::appserver-backend-config kubernetes:networking.gke.io/v1:ManagedCertificate urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:networking.gke.io/v1:ManagedCertificate::managed-cert kubernetes:core/v1:ConfigMap urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:ConfigMap::appserver kubernetes:apps/v1:Deployment urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:apps/v1:Deployment::nginx kubernetes:core/v1:Service urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:Service::nginx kubernetes:core/v1:Service urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:core/v1:Service::appserver gcp:dns/recordSet:RecordSet urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::gcp:dns/recordSet:RecordSet::ingress kubernetes:apps/v1:Deployment urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:apps/v1:Deployment::appserver kubernetes:networking.k8s.io/v1:Ingress urn:pulumi:gcp__us-west1__neara-latest::pulumi-k8s::kubernetes:networking.k8s.io/v1:Ingress::ingress

Found no pending operations associated with gcp__us-west1__neara-latest

Backend
Name sunburst URL s3://com-neara-bacek-test/pulumi/ User bacek Organizations

Dependencies: NAME VERSION @pulumi/gcp 6.43.0 @pulumi/kubernetes 3.22.1 @pulumi/kubernetesx 0.1.6 @pulumi/pulumi 3.46.0 @types/lodash 4.14.188 lodash 4.17.21 @types/node 14.18.33

Pulumi locates its logs in /tmp by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you’ve opened one already).

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 15 (6 by maintainers)

Commits related to this issue

Most upvoted comments

I’ve got a reproduction of this going locally now. Seems to be an issue where if a resource is replaced but fails to delete, then is replaced again we can assert (and also fail to correctly try deleting the resource again).

@aq17 I can try it. But pulumi refresh does go through. So state is not corrupted to the point where it’s totally useless.