azure-pipelines-agent: System.DefaultWorkingDirectory broken in version 2.161.1
Agent Version and Platform
Version of your agent? 2.161.1
OS of the machine running the agent? Ubuntu 16.04.6 LTS
Azure DevOps Type and Version
dev.azure.com
What’s not working?
It appears as if version 2.161.1 (or 2.161.0) broke the System.DefaultWorkingDirectory environment variable, at least in bash steps. When looking at its value inside a script, it is set to /data/azure/azure-agent/_work/10/s/, which is the path on the hosting machine. I believe it should be set to /__w/10/s, which is the path inside the agent. This used to work on 2.160.1, and our agents were updated overnight and multiple jobs stopped working.
We have a workaround, as it looks like the SYSTEM_DEFAULTWORKINGDIRECTORY environment variable is set properly, so we’ll update our scripts.
I’m also surprised that our agents were automatically updated by dev.azure.com despite that version being tagged as pre-release.
The broken step looked like:
node $(System.DefaultWorkingDirectory)/azure/update-versions.js
We were able to work around the bug by replacing it with:
node ${SYSTEM_DEFAULTWORKINGDIRECTORY}/azure/update-versions.js
Agent and Worker’s Diagnostic Logs
If there is a specific section of the log you are interested in, let me know and I’ll post it, but that log contains a lot of sensitive information, including our build steps, so I’m not willing to post it in it’s entirety.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 23 (10 by maintainers)
Commits related to this issue
- Fixed variable expansion for container targets #2646 — committed to microsoft/azure-pipelines-agent by deleted user 5 years ago
- Fixed variable expansion for container targets #2646 (#2649) * Fixed variable expansion for container targets #2646 * Updated dotnet-install scripts — committed to microsoft/azure-pipelines-agent by jtpetty 5 years ago
- Fixed string localization issue that ended up preventing the file path from being displayed #2646 — committed to microsoft/azure-pipelines-agent by deleted user 5 years ago
- Fixed multiple bugs caused by changes to support step targets #2646 (#2673) * Fixed string localization issue that ended up preventing the file path from being displayed #2646 * Fixed to convert p... — committed to microsoft/azure-pipelines-agent by jtpetty 5 years ago
@jbblanchet - 2.163.0 is now available on GitHub. You can manually upgrade to that version or you can wait for our rollout and it will do it automatically.
Also, I have triaged your failing case and am working on a fix. When I have something ready, I will message you and see how best to verify that this will no longer impact your team.
@jpricket thanks for that, we were able to make it work with your change yesterday.
Once that fire is extinguished, I think there’s a larger conversation to have about giving users control over the version upgrades. It’s not only that the new version broke something. It’s that it broke something at a very bad moment for us, and that we had no way of rolling back the changes.
We’re affected by this as well. There’s a community thread: https://developercommunity.visualstudio.com/content/problem/844025/container-build-jobs-paths-are-incorrectly-mapped.html
Our builds were working fine on 2.160.1, so it looks like 2.161.1 is the culprit.
Our agents are automatically upgrading to a release that is marked as a pre-release - I’m not sure if that’s the expected behavior. Also I’m not sure if there is a way to disable automatic upgrades as that would allow us to downgrade.