azure-pipelines-tasks: Bash 'task.setvariable' adds unexpected ' as trailing symbol

Type: Bug

Task Name: Bash

Environment

Server - Azure Pipelines
Agent - Private:
- OS: Red Hat Enterprise Linux Server release 7.4 (Maipo)
- Agent version: 2.141.1

Issue Description

When echo "##vso[task.setvariable variable=variable]$value" is used as a part of script in Bash task type some unexpected trailing ' may appear in the value of variable. Appearance of trailing ' is a fully random process, but I usually get them in the part of my variables.

To produce and show appearance of trailing ' in value of variables I set two variables: build.producedArtifactName and build.producedArtifactPath, as could be seen below, I work with this variables in the same style: use some bash commands to set values and expose them using ###vso set.variable, in next Tasks I echo this variables and use them as an input for Task, as could be seen on screenshots, trailing ' appeared in value of variable build.producedArtifactName and didn’t for build.producedArtifactPath that makes me think that problem came from the side of Bash V3 module.

Here simple bash script sets two variables and uses ###vso for variables setting:

As it could be seen in log, set variables don’t have any ' symbols: Result of simple bash script

Next, it’s used simple bash script to echo variables which were set via ###vso: Simple bash script for new vars

As a result output for first variable absolutely normal, but in second variable trailing ' appeared: Appearance of trailing symbol

If variable set via ###vso is used in other types of tasks like Publish Build Artifacts same behaviour is faced: Test variable with other task types

As a result drop artifact also has trailing ' in it’s name: Result for other task type

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 3
Comments: 24

Commits related to this issue

Avoid using `set -x` in azure pipeline This hopefully works around microsoft/azure-pipelines-tasks/issues/10331 — committed to tomtung/omikuji by tomtung 5 years ago
CI: remove set -x - An extra "'" may be added to the day - https://github.com/microsoft/azure-pipelines-tasks/issues/10331 [skip ci] — committed to cctbx/cctbx_project by bkpoon 4 years ago
Fix unexpected trailing symbol (#6545) co-authorized by: jianquanye@microsoft.com * Fix unexpected trailing symbol Approach What is the motivation for this PR? With set -x, "##vso[task.setvaria... — committed to sonic-net/sonic-mgmt by yejianquan 2 years ago
Fix unexpected trailing symbol (#6545) co-authorized by: jianquanye@microsoft.com * Fix unexpected trailing symbol Approach What is the motivation for this PR? With set -x, "##vso[task.setvaria... — committed to allen-xf/sonic-mgmt by yejianquan 2 years ago
Add explicit 'set +x' before printing a vso[] command to avoid output getting parsed again with a trailing quote. (#15986) Here's the motivating issue: https://github.com/microsoft/azure-pipelines-t... — committed to microsoft/onnxruntime by edgchen1 a year ago

Most upvoted comments

Hello, @damccorm. As participants above absolutely truly mentioned, this issue occurs only when you set set -x in bash tasks. Also probability to reproduce issue is very random. For some tasks it occurs every run, for another it may not be occurred at all. Various experiments with quoting haven’t gave any result at all. In our case we had to remove all mentions of set -x from our bash tasks, it’s very sad, because Azure Pipelines has very poor diagnostic of runtime execution of bash scripts. As I can see, since my last visit demand on this issue grown so mush, so I hope that bugfix will be released someday.

Just a little bit of information for future visitors of this issue: As we recognised in the process of our internal investigation, Azure Pipelines just scans output of all commands in bash script and if sees special command like ##vso, then executes it immediately even if it’s just an output of an -x option (you may notice that if your echo or set -x output has ##vso command it won’t be shown at logging screen at all). As set -x option may add some extra single quotes to debugging output (starting with + symbol) , Azure Pipelines identifies this extra quotes as a part of variable that should be set. Also task.setvariable may set value of variable only once, so when echo command provide clear output without any quotes to stdout it won’t change value which was already set using output from set -x option. As a workaround set +x may be added before calls of ##vso functions, it may help if you need to have output of set -x options for some parts of your code but such output for ##vso functions may be suppressed.

nuclear0wl on Oct 29, 2019

TL;DR: having “set -x” with echo vso vars will add trailing apostrophes.

mhvelplund on Feb 9, 2023

I’m going to close since I think this is a no-op on our end. Please comment if you think it should be reopened

damccorm on Nov 20, 2019

I guess we made the (incorrect) assumption that ##vso has to appear at the beginning of a line to be processed.

But isn’t it a reasonable assumption? a) Consistency: It picks up everything until the end of the line, so it should start at the beginning. b) Consistency: If you parse bash output in the bash world, you would not see the Processed: prefix. c) The documentation gives only examples which start at the beginning of the line. d) Security: No string echoed to the console by some command should interfer with the environment too easily.

dg0yt on Nov 14, 2019