pulumi: Access failure after upgrading to 2.23.1 using azure blob storage as state backend

After upgrading from 2.22.0 to 2.23.1 I get an error when Pulumi tries to access Azure blob storage for state. Rollbacking to 2.22.0 makes it work once again.

error: saving update info: blob (key ".pulumi/stacks/testsa342342.json -> .pulumi/history/testsa342342/testsa342342-1616410081849968900.checkpoint.json") (code=Unknown): -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /Users/runner/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.13.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationPermissionMismatch) =====
Description=This request is not authorized to perform this operation using this permission.
RequestId:1c982d00-f01e-0033-6008-1f8593000000
Time:2021-03-22T10:48:00.8339304Z, Details:
   Code: AuthorizationPermissionMismatch
   PUT https://testsa342342.blob.core.windows.net/pulumi-state/.pulumi/history/testsa342342/testsa342342-1616410081849968900.checkpoint.json?se=2030-03-23t10%3A22%3A00z&sig=REDACTED&sp=racwdxl&sr=c&st=2021-03-20t10%3A22%3A00z&sv=2020-04-08&timeout=61
   User-Agent: [go-cloud/blob/0.1.0 Azure-Storage/0.13 (go1.16.2; linux)]
   X-Ms-Client-Request-Id: [cab8237f-1c29-4ad8-4eab-c1e448300257]
   X-Ms-Copy-Source: [https://testsa342342.blob.core.windows.net/pulumi-state/.pulumi/stacks/testsa342342.json?se=2030-03-23t10%3A22%3A00z&sig=REDACTED&sp=racwdxl&sr=c&st=2021-03-20t10%3A22%3A00z&sv=2020-04-08]
   X-Ms-Tags: []
   X-Ms-Version: [2019-12-12]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation using this permission.
   Content-Length: [279]
   Content-Type: [application/xml]
   Date: [Mon, 22 Mar 2021 10:48:00 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [cab8237f-1c29-4ad8-4eab-c1e448300257]
   X-Ms-Error-Code: [AuthorizationPermissionMismatch]
   X-Ms-Request-Id: [1c982d00-f01e-0033-6008-1f8593000000]
   X-Ms-Version: [2019-12-12]



ERROR: 255

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 6
Comments: 17 (8 by maintainers)

Commits related to this issue

Use fork of go-cloud to temporarily unblock #6596 — committed to pulumi/pulumi by deleted user 3 years ago

Most upvoted comments

Hi friends,

I am happy to say that we just released v2.25.2 of Pulumi which fixes up this issue

Before:

~/code/test-azure-backend
▶ pulumi version
v2.25.1


▶ PULUMI_CONFIG_PASSPHRASE=password pulumi up
Please choose a stack, or create a new one: dev
Previewing update (dev):
     Type                 Name                    Plan
     pulumi:pulumi:Stack  test-azure-backend-dev

Resources:
    2 unchanged

Do you want to perform this update? yes
Updating (dev):
     Type                 Name                    Status
     pulumi:pulumi:Stack  test-azure-backend-dev

Resources:
    2 unchanged

Duration: 2s

error: saving update info: blob (key ".pulumi/stacks/dev.json -> .pulumi/history/dev/dev-1618666697398562000.checkpoint.json") (code=Unknown): -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.13.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthorizationPermissionMismatch) =====
Description=This request is not authorized to perform this operation using this permission.
RequestId:7ca940da-c01e-00a6-178e-3365b1000000
Time:2021-04-17T13:38:17.7146784Z, Details:
   Code: AuthorizationPermissionMismatch
   PUT https://stack72testing.blob.core.windows.net/statestore/.pulumi/history/dev/dev-1618666697398562000.checkpoint.json?se=[redacted]=2020-02-10&timeout=61
   User-Agent: [go-cloud/blob/0.1.0 Azure-Storage/0.13 (go1.16.3; darwin)]
   X-Ms-Client-Request-Id: [66cce542-44d0-4096-5845-fe2246aced7e]
   X-Ms-Copy-Source: [https://stack72testing.blob.core.windows.net/statestore/.pulumi/stacks/dev.json?se=[redacted]=2020-02-10]
   X-Ms-Tags: []
   X-Ms-Version: [2019-12-12]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 This request is not authorized to perform this operation using this permission.
   Content-Length: [279]
   Content-Type: [application/xml]
   Date: [Sat, 17 Apr 2021 13:38:17 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Client-Request-Id: [66cce542-44d0-4096-5845-fe2246aced7e]
   X-Ms-Error-Code: [AuthorizationPermissionMismatch]
   X-Ms-Request-Id: [7ca940da-c01e-00a6-178e-3365b1000000]
   X-Ms-Version: [2019-12-12]

After:

~/code/test-azure-backend
▶ pulumi version
v2.25.2

~/code/test-azure-backend
▶ PULUMI_CONFIG_PASSPHRASE=password pulumi up
Previewing update (dev):
     Type                 Name                    Plan
     pulumi:pulumi:Stack  test-azure-backend-dev

Resources:
    2 unchanged

Do you want to perform this update? yes
Updating (dev):
     Type                 Name                    Status
     pulumi:pulumi:Stack  test-azure-backend-dev

Resources:
    2 unchanged

Duration: 2s

stack72 on Apr 17, 2021

AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_SAS_TOKEN

WillooWisp on Apr 1, 2021

Based on the changelog, it sounds like this may be caused by #2697 somehow? cc @lukehoban

None of the code in that PR will run unless an env var is set. I strongly suspect https://github.com/pulumi/pulumi/pull/6473 is the actual trigger for this. cc @stack72.

lukehoban on Mar 31, 2021

I can confirm this issue in 2.23.1 and working around it by downgrading to 2.22.0

aadje on Mar 23, 2021