azure-pipelines-tasks: Cannot use parameter / variables for 'azureSubscription' in AzurePowerShell task in deployment

Running into this exact same issue. I’m trying to set the value for azureResourceManagerConnection using the matrix strategy because I need to test the same steps for multiple subscriptions but my runs fail because we can’t do that. Assigning a global variable is also not a decent hack because I would need to hardcode in multiple tasks in the same pipeline which is a big pain to manager.

@AmrutaKawade / @nadesu any estimated ETA for the fix for this?

This is an issue as well for us. What is a simple way to utilize a variable in a -job does not work in a -deployment

Will have to use a work-around approach. This 100% is a bug.

The simple fact that these tasks like AzureFunctionApp cannot accept a variable for input on the param azureSubscription, but can for the appName… is simply a defect.

https://developercommunity.visualstudio.com/t/using-a-variable-for-the-service-connection-result/676259

~ THREE YEARS OLD AND STILL NOT FIXED ~

I got an email that someone figured something out - but came to login to review it and the post isn’t here?

Maybe it didn’t work?

Pretty sure they aren’t going to fix this. It’s been an issue for a long time, and it’s just not a priority.

Hit this issue. I am not using templates. Have a single azure-pipelines.yml with a stage that has a deployment job. Adding a stage or job level variable, and then trying to set connectedServiceName referencing that variable:

connectedServiceName: '$(connectionName)'

doesn’t work.

I have to use a literal:

connectedServiceName: 'ARM - env-dev'

This is extremely odd. Usually you can use variables to set properties of tasks.

Use a parameter. It’s the simplest way of doing this. You don’t need to hardcode variables in each step. Pass in the subscription name as a parameter from a root yaml file and you don’t need to hardcode anything.

I do it like so:

trigger: none
extends:
  template: /Main-Template.yml
  parameters:
    buildVariableGroup: 'Some_Build_VarGroup'
    environments:
      - environment: QA
        variableGroup: QA_Var_Group        
        deploySvcConnection: "QA_Connection'

Here I have build & deploy stages in “Main-Template.yml” where I loop through environments for the deploy stage. Both Build & Deploy stage call further templates. My root yaml file, as shown above, actually has nothing BUT parameters.

You can use ${{ variables.serviceconnection }}. But this practice means that you have to specify variables in advance (Before you run the pipeline).

For service connections, you can specify a value directly or use the ’compile-time variable‘ xxx,whichwillexpandandthenpopulatetheserviceconnectionsectionwithvaluesbeforerunning.Inthisusageof(xxx), the service connection of the task cannot be obtained, because this is a runtime value. The service connection needs to be specified before running. The changes (runtime changes) of the variables during the pipeline run will not be acquired by the service connection part of the subsequent task.

You are using a runtime variable.

But run time variables aren’t supported for service connection OR azure subscription. The variable will get initialized at the run time. https://github.com/microsoft/azure-pipelines-tasks/issues/10376#issuecomment-514477023

Parameters are expanded just before the pipeline runs. This is by design.

Also clearly in this official document: https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#use-a-service-connection

The entire point of this bug and thread and on-going discussion is that you have to hard-code the subscription name. The people here are trying to build complex pipelines that can utilize multiple subscriptions with logic determining which subscription.

We don’t want to hard code those - should the subscription change. (e.g. one sub for Dev, one sub for Test, one sub for Prod )

@ForteUnited we now paramerize the service connection names - one is unable to pass it as a variable… but can make it a parameter… we choose to do this within the environment parameter - giving us the ability to easily switch service connections for different environments - example… within my root pipeline i’ll have something like

extends: template: /SomePath/My_template.yml parameters: environments: - environment: QA variableGroup: Variables-QA AzureServiceConnection: ‘Dev_Tenant’ - environment: UAT variableGroup: Variables-UAT AzureServiceConnection: ‘Dev_Tenant’ - environment: PRODUCTION variableGroup: Variables-PROD AzureServiceConnection: ‘Production_Tenant’

And in the templates I refer to is as so: azureSubscription: ${{ environment.AzureServiceConnection }}

Pasting my yaml here ruined all the indenting but you can get the general idea.

This is still an issue. But i have a workaround that is quite simple. use a parameter inside the template that does not have a default.

The problem If you provide the azureSubscription input in an AzureRmWebAppDeployment@4 that is used inside a deployment job inside a template, passing it as a parameter it fails. it does not expand the parameter.

However it is a bit more specific. Parameters that have no default are expanded correctly. so in a template.yml file:

parameters:
  subscription:

does work when using this parameter in the AzureRmWebAppDeployment@4 azuresubscription input field. But what fails is if it is passed with a default value:

parameters:
  - subscription: environment
    type: string
    default: 'default'

the error message when saving it in to git from the edit pipeline screen results in a message saying that the azureSubscription ‘default’ does not exist.

Can this please be fixed? All other parameters within the - deployment: job are expanded correctly but for the azureSubscription input of the AzureRmWebAppDeployment@4 it fails.

I think there is a validation that checks if a valid azureSubscription is used. But when checking this it takes the default if available instead of the expanded parameter.

here’s what i ended up doing, yuck

- name: serviceConnection
  value: $[replace(replace(eq(variables['environment'],'prod'), 'True', 'prod-connection'), 'False', 'other-connection')]

this is outside the stage. you could use in() instead of eq if you wanted to match multiple. Also instead of other-connection you could do yet another chain like this

- name: serviceConnection
  value: $[replace(replace(replace(eq(variables['environment'],'prod'), 'True', 'prod-connection'), 'False', replace(eq(variables['environment'],'stage'), 'True', 'stage-connection'), 'False', 'other-connection-not-prod-or-stage')]

i feel dirty now and need to take a shower

If it helps for anyone else running into this issue, I did a dirty hack to get this working by implementing this workaround in a generic way. I run this code here to replace the subscription passed in the with the correct subscription

 - ${{ each step in parameters.StepList }}:
     - ${{ each pair in step }}:
        ${{ if ne(pair.key, 'parameters') }}:
          ${{ pair.key }}: ${{ pair.value }}
        ${{ if eq(pair.key, 'parameters') }}:
          parameters:
            ${{ each input in pair.value }}:
                ${{ if eq(input.key, 'AzureSubscription') }}:
                  AzureSubscription: ${{ parameters.stage.azuresubscription }}
                ${{ if ne(input.key, 'AzureSubscription')  }}:
                  ${{ input.key }}: ${{ input.value }}

Hi,

Yes, it will be workaround.

But it is still a bug: we cannot use the variables from the library to choose subscription.

And therefore we cannot use the same template across multiple pipelines that deploys to different subscriptions that are set on given Azure DevOps projects.

I found interesting behaviour. We keep Service Connection names in a variable group.

Option 1 (works): Single Service Connection

Read service connection name on the root level before going through all stages.

variables:
  - group: 'AzureResources'
  - name:  pipelineServiceConnection
    value: $[variables['${{ parameters.productName }}_pipelineServiceConnection']]
stages:
  - ${{ each stage in parameters.stages }} :

    - task: AzurePowerShell@5
              inputs: 
                azureSubscription: $(pipelineServiceConnection)

Option 2 (doesn’t work): Multiple Service Connections

Required to have separate service connection for each stage-environment pair Read service connection name stage or job level

  - ${{ each stage in parameters.stages }} :
      - stage: ${{ stage.stage }}
        displayName: Deploy ${{ stage.stage }}
        dependsOn: ${{ stage.dependsOn }}

        jobs:
        - deployment: Deploy_${{ stage.stage }}_${{ parameters.productName }}
          variables:
          - group: 'AzureResources'
          - name: pipelineServiceConnection
            value: $[variables['${{ parameters.productName }}_pipelineServiceConnection_${{ stage.stage }}']]

          strategy:
            runOnce:
              deploy:
                steps:
                  - task: AzurePowerShell@5
                    inputs:
                      azureSubscription:  $(pipelineServiceConnection)  

Tried azureSubscription: ${{ variables.pipelineServiceConnection }} doesn’t work either. Always got errors

“service connection $(pipelineServiceConnection) which could not be found” or “service connection $[variables[‘yamlLearning_pipelineServiceConnection’]] which could not be found”

I guess I cannot pass validation.

Option 3 (works): Var Template Files for Multiple Service Connections

I don’t understand why it works with var template files, but doesn’t with variable groups.

        variables:
          - name:  environment
            value: ${{ stage.stage }}
          - template: ../vars/environment.${{variables['environment']}}.yaml

Template expression works azureSubscription: ${{ variables.pipelineServiceConnection }}

===== and finally =====

Option 4 (works interesting way)

We don’t want to keep service connections in the code. So I’ve registered a dummy service connection and on the root level assigned to the var. Now it works on stage or job level

variables:
  - name:  pipelineServiceConnection
    value: 'DevOps-AscArm-Dummy'

  - ${{ each stage in parameters.stages }} :
      - stage: ${{ stage.stage }}
        displayName: Deploy ${{ stage.stage }}
        dependsOn: ${{ stage.dependsOn }}

        jobs:
        - deployment: Deploy_${{ stage.stage }}_${{ parameters.productName }}
          variables:
          - group: 'AzureResources'
          - name: pipelineServiceConnection
            value: $[variables['${{ parameters.productName }}_pipelineServiceConnection_${{ stage.stage }}']]

          strategy:
            runOnce:
              deploy:
                steps:
                  - task: AzurePowerShell@5
                    inputs:
                      azureSubscription:  $(pipelineServiceConnection)  

Can someone explain why it works when the variable overwritten on stage or job level?

Let’s beat a dead horse… Using a variable for the azureSubscription input works with a Release Definition. So essentially we are being told that Microsoft broke this on purpose when they started pushing YAMl pipelines. Additionally this is severely short sighted not to implement as GitHub Actions and GitLab both can support this.

I know it is more simple than but, wouldn’t it just be a new version of the AzPowerShell and AzCLI tasks that allow for runtime variables? Or at minimum support recursive parsing after all variable groups and variable files are loaded within a stage/job?

/rant

Use a parameter. It’s the simplest way of doing this. You don’t need to hardcode variables in each step. Pass in the subscription name as a parameter from a root yaml file and you don’t need to hardcode anything.

I do it like so:

trigger: none

extends:

template: /Main-Template.yml

parameters:

buildVariableGroup: 'Some_Build_VarGroup'

environments:

  - environment: QA

    variableGroup: QA_Var_Group        

    deploySvcConnection: "QA_Connection'

Here I have build & deploy stages in “Main-Template.yml” where I loop through environments for the deploy stage. Both Build & Deploy stage call further templates. My root yaml file, as shown above, actually has nothing BUT parameters.

You can use ${{ variables.serviceconnection }}. But this practice means that you have to specify variables in advance (Before you run the pipeline).

For service connections, you can specify a value directly or use the ’compile-time variable‘ xxx,whichwillexpandandthenpopulatetheserviceconnectionsectionwithvaluesbeforerunning.Inthisusageof(xxx), the service connection of the task cannot be obtained, because this is a runtime value. The service connection needs to be specified before running. The changes (runtime changes) of the variables during the pipeline run will not be acquired by the service connection part of the subsequent task.

You are using a runtime variable.

But run time variables aren’t supported for service connection OR azure subscription. The variable will get initialized at the run time. https://github.com/microsoft/azure-pipelines-tasks/issues/10376#issuecomment-514477023

Parameters are expanded just before the pipeline runs. This is by design.

Also clearly in this official document: https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#use-a-service-connection

The entire point of this bug and thread and on-going discussion is that you have to hard-code the subscription name. The people here are trying to build complex pipelines that can utilize multiple subscriptions with logic determining which subscription.

We don’t want to hard code those - should the subscription change.

(e.g. one sub for Dev, one sub for Test, one sub for Prod )

You were faster than me to answer the previous comment. 😉

Use a parameter. It’s the simplest way of doing this. You don’t need to hardcode variables in each step. Pass in the subscription name as a parameter from a root yaml file and you don’t need to hardcode anything.