azure-pipelines-tasks: [BUG]: AzureFileCopy@5 cleanup fails in GetJobResults with "failed to parse user input due to error: cannot use wildcards in the path section of the URL..."

New issue checklist

Task name

AzureFileCopy

Task version

5.234.0

Issue Description

The task began to fail today with the error message: failed to parse user input due to error: cannot use wildcards in the path section of the URL except in trailing "/*". If you wish to use * in your URL, manually encode it to %2A We only have a trailing slash in SourcePath in the command definition, but from my inspection, it looks like it’s failing because the blob storage URL has 5 asterisks appended to the end. Those asterisks are not something I’ve input into the task, and they aren’t part of the UUID, as you can see from earlier in the log

Environment type (Please select at least one enviroment where you face this issue)

Self-Hosted
Microsoft Hosted
VMSS Pool
Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Windows 11

Relevant log output

2024-02-08T20:33:01.1792738Z ##[debug][Azure Call]Creating container: 99c69625-ee4a-45f7-86a0-11bcfc92588b in storage account: 3dtmbinarystorage
2024-02-08T20:33:02.7981716Z ##[debug][Azure Call]Created container: 99c69625-ee4a-45f7-86a0-11bcfc92588b successfully in storage account: 3dtmbinarystorage
...
contact me directly @microsoft for full log if necessary
...
2024-02-08T20:33:28.4016542Z VERBOSE: Exporting function 'New-PSWorkflowSession'.
2024-02-08T20:33:28.4033884Z VERBOSE: Exporting alias 'nwsn'.
2024-02-08T20:33:29.0328279Z ##[debug]JobId: '2', JobState: 'running', ComputerName: '$(WSUSVMFQDN)'
2024-02-08T20:33:29.0352474Z ================================================ $(WSUSVMFQDN) ================================================
2024-02-08T20:33:29.1798996Z ##[command]& 'C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe' -NoLogo -NoProfile -NonInteractive 
2024-02-08T20:33:29.1800960Z -ExecutionPolicy Unrestricted -Command ". 'C:\Users\3DTM\AppData\Local\Temp\564badbc-2273-449c-b27a-c207ee9d8467.ps1'"
2024-02-08T20:33:29.1801744Z 
2024-02-08T20:33:34.2738043Z ##[debug]JobId: '2', JobState: 'completed', ComputerName: '$(WSUSVMFQDN)'
2024-02-08T20:33:34.2762788Z ================================================ wsus-3dtm.uksouth.cloudapp.azure.com ================================================
2024-02-08T20:33:34.2825861Z VERBOSE: Using default AzCopy arguments for dowloading to VM
2024-02-08T20:33:34.2829285Z 
2024-02-08T20:33:34.3144160Z VERBOSE: ##[command] & azcopy copy 
2024-02-08T20:33:34.3147566Z 
2024-02-08T20:33:34.3635588Z "https://$(StorageResourceURL)/99c69625-ee4a-45f7-86a0-11bcfc92588b*****" 
2024-02-08T20:33:34.3638378Z 
2024-02-08T20:33:34.3925065Z "E:\MSUS\WsusContent\TRANSFER" --recursive --log-level=INFO
2024-02-08T20:33:34.3928494Z 
2024-02-08T20:33:34.4195491Z 
2024-02-08T20:33:34.4199248Z 
2024-02-08T20:33:34.4472492Z failed to parse user input due to error: cannot use wildcards in the path section of the URL except in trailing "/*". If you wish to use * in your URL, manually encode it to %2A
2024-02-08T20:33:34.4475185Z 
2024-02-08T20:33:34.4752474Z ##[debug][3DTM-Glasgow-WS]LASTEXITCODE is 1
2024-02-08T20:33:34.4754589Z 
2024-02-08T20:33:34.4902948Z ##[debug]Leaving Get-JobResults.
2024-02-08T20:33:34.4992830Z ##[debug]Leaving Run-RemoteScriptJobs.

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION]

Repro steps

steps:
- task: AzureFileCopy@5
  displayName: 'Copy 3DTM Launcher Service to WSUS'
  inputs:
    SourcePath: '"C:\Telemedicine3DOneDrive\OneDrive\FullSystemDeploy\$(Build.SourceBranchName)\3DTMLauncherService.*"'
    azureSubscription: 'Peabody Pipeline upload'
    Destination: AzureVMs
    storage: 3dtmbinarystorage
    resourceGroup: 3DTELEMEDECINE
    MachineNames: '$(WSUSVMName)'
    vmsAdminUserName: '$(WSUSAdminUsername)'
    vmsAdminPassword: '$(WSUSAdminPassword)'
    TargetPath: '$(WSUSDestinationFolder)'

About this issue

Original URL
State: open
Created 5 months ago
Reactions: 3
Comments: 28

Most upvoted comments

If it helps, I worked around it like this:

Change the Destination Type of the Azure File Copy task to Azure Blob

Put something in Container Name like “foo”

Check the Clean Target checkbox

Under Output Variables, put something in Reference Name like “bar”

Add a PowerShell on Target Machines task

Enter the server name of your target machine and login credentials

Add the following as an Inline Script: azcopy copy “https:////*?$(.StorageContainerSasToken)” “” --recursive --log-level=INFO

The reconfigured Azure File Copy task will do the “copy to blob” bit, and the new task will do the “copy from blob to target” bit.

Thanks, couple of questions - is the output variable “bar” used anywhere - is that how the azcopy knows which storage container to copy? is the StorageContainerSasToken automatically set to the correct value?

Sorry, updated my original comment. I forgot to escape the angle brackets.

haf76348g78 on Mar 19, 2024

If it helps, I worked around it like this:

Change the Destination Type of the Azure File Copy task to Azure Blob
Put something in Container Name like “foo”
Check the Clean Target checkbox
Under Output Variables, put something in Reference Name like “bar”
Add a PowerShell on Target Machines task
Enter the server name of your target machine and login credentials
Add the following as an Inline Script: azcopy copy "https://<your storage>/<foo>/*?$(<bar>.StorageContainerSasToken)" "<your destination>" --recursive --log-level=INFO

The reconfigured Azure File Copy task will do the “copy to blob” bit, and the new task will do the “copy from blob to target” bit.

haf76348g78 on Mar 19, 2024

Copying from the blob to the VM is still working as of now if azcopy is invoked from the VM using the PowerShellOnTargetMachines@3 task

volaru77 on Mar 26, 2024

I’m working around this by downloading the SXS version of this task…fixed the bad syntax in the power shell file…the azcopy command for copying from the storage account to the VM needed a question mark character between the container and the SAS params.

Experiencing the same problem today, but was working yesterday and before. Same root cause (missing question mark). Don’t understand how this could suddenly start happening today?

haf76348g78 on Mar 19, 2024

I’m working around this by downloading the SXS version of this task…fixed the bad syntax in the power shell file…the azcopy command for copying from the storage account to the VM needed a question mark character between the container and the SAS params.

we shouldn’t have to work around these sorts of bugs all the time.

jiggybyte on Feb 23, 2024

Same error…how are we expected to use the V5 task to remove dependencies on AzureRM if V5 is broken? Classic MS.

jiggybyte on Feb 22, 2024