pulumi-terraform-bridge: Error on creating azure.storage.ManagementPolicy

Please let me know if there’s a remaining issue blocking your work and we’ll prioritized accordingly; if it’s just an annoying quirk then we might need to park for now; when coming back to it it would be great to cross-compare with how TF CLI handles this program and figure out these details.

Well, I can sort of reproduce the -1 appearing but I’m wondering if this is a problem. Here’s a program I’m using.

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure";

const containerName = "my-container";

const exampleResourceGroup = new azure.core.ResourceGroup("exampleResourceGroup", {location: "West Europe"});

const exampleAccount = new azure.storage.Account("exampleaccount", {
    resourceGroupName: exampleResourceGroup.name,
    location: exampleResourceGroup.location,
    accountTier: "Standard",
    accountReplicationType: "GRS",
    tags: {
        environment: "staging",
    },
});

export const managementPolicy = new azure.storage.ManagementPolicy("managementPolicyRobotFramework", {
    storageAccountId: exampleAccount.id,
    rules: [
      {
        name: "robot-framework",
        enabled: true,
        filters: {
          prefixMatches: [
            containerName,
          ],
          blobTypes: [
            "blockBlob",
          ],
        },
        actions: {
          baseBlob: {
            deleteAfterDaysSinceModificationGreaterThan: 30
          },
          snapshot: {
            deleteAfterDaysSinceCreationGreaterThan: 30,
          },
          version: {
            deleteAfterDaysSinceCreation: 30
          }
        }
      }
    ]
  })

pulumi up succeeds.

There are some -1s showing indeed.

The first one shows up in Check because the upstream spec for it does not specify ValidateFunc range for some reason, so tierToCoolAfterDaysSinceLastAccessTimeGreaterThan=-1 is a valid value as far as the provider cares.

{
  "method": "/pulumirpc.ResourceProvider/Check",
  "request": {
    "urn": "urn:pulumi:dev::repro-pulumi-terraform-bridge-720::azure:storage/managementPolicy:ManagementPolicy::managementPolicyRobotFramework",
    "olds": {},
    "news": {
      "rules": [
        {
          "actions": {
            "baseBlob": {
              "deleteAfterDaysSinceModificationGreaterThan": 30
            },
            "snapshot": {
              "deleteAfterDaysSinceCreationGreaterThan": 30
            },
            "version": {
              "deleteAfterDaysSinceCreation": 30
            }
          },
          "enabled": true,
          "filters": {
            "blobTypes": [
              "blockBlob"
            ],
            "prefixMatches": [
              "my-container"
            ]
          },
          "name": "robot-framework"
        }
      ],
      "storageAccountId": "04da6b54-80e4-46f7-96ec-b56ff0331ba9"
    },
    "randomSeed": "xOM+qoIZSxQzKzUdSAeu78Cj1UktgyhchureF37Pxx0="
  },
  "response": {
    "inputs": {
      "__defaults": [],
      "rules": [
        {
          "__defaults": [],
          "actions": {
            "__defaults": [],
            "baseBlob": {
              "__defaults": [
                "tierToCoolAfterDaysSinceLastAccessTimeGreaterThan"
              ],
              "deleteAfterDaysSinceModificationGreaterThan": 30,
              "tierToCoolAfterDaysSinceLastAccessTimeGreaterThan": -1
            },
            "snapshot": {
              "__defaults": [],
              "deleteAfterDaysSinceCreationGreaterThan": 30
            },
            "version": {
              "__defaults": [],
              "deleteAfterDaysSinceCreation": 30
            }
          },
          "enabled": true,
          "filters": {
            "__defaults": [],
            "blobTypes": [
              "blockBlob"
            ],
            "prefixMatches": [
              "my-container"
            ]
          },
          "name": "robot-framework"
        }
      ],
      "storageAccountId": "04da6b54-80e4-46f7-96ec-b56ff0331ba9"
    }
  },
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "azure"
  }
}

Things get a lot more interesting in Create preview.

{
  "method": "/pulumirpc.ResourceProvider/Create",
  "request": {
    "urn": "urn:pulumi:dev::repro-pulumi-terraform-bridge-720::azure:storage/managementPolicy:ManagementPolicy::managementPolicyRobotFramework",
    "properties": {
      "__defaults": [],
      "rules": [
        {
          "__defaults": [],
          "actions": {
            "__defaults": [],
            "baseBlob": {
              "__defaults": [
                "tierToCoolAfterDaysSinceLastAccessTimeGreaterThan"
              ],
              "deleteAfterDaysSinceModificationGreaterThan": 30,
              "tierToCoolAfterDaysSinceLastAccessTimeGreaterThan": -1
            },
            "snapshot": {
              "__defaults": [],
              "deleteAfterDaysSinceCreationGreaterThan": 30
            },
            "version": {
              "__defaults": [],
              "deleteAfterDaysSinceCreation": 30
            }
          },
          "enabled": true,
          "filters": {
            "__defaults": [],
            "blobTypes": [
              "blockBlob"
            ],
            "prefixMatches": [
              "my-container"
            ]
          },
          "name": "robot-framework"
        }
      ],
      "storageAccountId": "04da6b54-80e4-46f7-96ec-b56ff0331ba9"
    },
    "preview": true
  },
  "response": {
    "properties": {
      "id": "",
      "rules": [
        {
          "actions": {
            "baseBlob": {
              "deleteAfterDaysSinceCreationGreaterThan": -1,
              "deleteAfterDaysSinceLastAccessTimeGreaterThan": -1,
              "deleteAfterDaysSinceModificationGreaterThan": 30,
              "tierToArchiveAfterDaysSinceCreationGreaterThan": -1,
              "tierToArchiveAfterDaysSinceLastAccessTimeGreaterThan": -1,
              "tierToArchiveAfterDaysSinceLastTierChangeGreaterThan": -1,
              "tierToArchiveAfterDaysSinceModificationGreaterThan": -1,
              "tierToCoolAfterDaysSinceCreationGreaterThan": -1,
              "tierToCoolAfterDaysSinceLastAccessTimeGreaterThan": -1,
              "tierToCoolAfterDaysSinceModificationGreaterThan": -1
            },
            "snapshot": {
              "changeTierToArchiveAfterDaysSinceCreation": -1,
              "changeTierToCoolAfterDaysSinceCreation": -1,
              "deleteAfterDaysSinceCreationGreaterThan": 30,
              "tierToArchiveAfterDaysSinceLastTierChangeGreaterThan": -1
            },
            "version": {
              "changeTierToArchiveAfterDaysSinceCreation": -1,
              "changeTierToCoolAfterDaysSinceCreation": -1,
              "deleteAfterDaysSinceCreation": 30,
              "tierToArchiveAfterDaysSinceLastTierChangeGreaterThan": -1
            }
          },
          "enabled": true,
          "filters": {
            "blobTypes": [
              "blockBlob"
            ],
            "matchBlobIndexTags": [],
            "prefixMatches": [
              "my-container"
            ]
          },
          "name": "robot-framework"
        }
      ]
    }
  },
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "azure"
  }
}

For things like changeTierToArchiveAfterDaysSinceCreation under version -1 seems illegal. But that somehow works out now.

Reopening this at the request of the requestor, please triage and close again if you need to.

Thank you for this new bit of info! In my understanding Pulumi actually does run TF state migrations in principle so we’ll need to take a closer look here.

I have also this pinned in a unit test now in https://github.com/pulumi/pulumi-terraform-bridge/pull/874 which makes it easier to experiment.

Unfortunately poking around it I’m still bit confused of where a fix should go, if anywhere. At first I thought that the issue is that the previous fix didn’t go “deep enough” and some bridge code got exposed to pre-defaults-correction schema. But it looks like that’s not sufficient. Something around here:

func (s v2InstanceState) Object(sch shim.SchemaMap) (map[string]interface{}, error) {

is calling into schema.FieldReader that’s not seeing corrections.

Moreover, just playing with correcting the schema by hand, I was not able to coax it to just skip the attributes like “tierToArchiveAfterDaysSinceLastAccessTimeGreaterThan”: -1 for some reason… If not given default values -1, it populates them with 0, even though they’re optional. 0 is probably not what we want at all since provider may take action as if it’s a constraint 0 (not missing).

Thank you for a detailed issue!

I have a fix for consideration that works around this class of error by ignoring defaults that do not validate:

https://github.com/pulumi/pulumi-terraform-bridge/pull/768

I have confirmed it fixes the problem as stated when applied to pulumi/pulumi-azure.

The other suggestions are interesting but given a large number of providers that depend on the bridge that may rely on the current behavior currently there is a strong preference for tactical fixes.

Transfering to the bridge project for further investigation on how this could be addressed without regressions.

@danielrbradley As I stated in my analysis, I’d throw out the current handling (enrichment) of default values out of the window in TF Bridge. But I don’t know what consequences this would imply.

This analysis looks good to me. I sounds like the core issue resides in the bridging code. Will raise internally to look at this case.