azure-pipelines-tasks: Azure file copy fails to copy to Azure Blob
Note
Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo
For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
If you have an issue or request for the Azure Pipelines service, use developer community instead:
https://developercommunity.visualstudio.com/spaces/21/index.html )
Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: Azure file copy
Environment
-
Server - Azure Pipelines
-
Account name - dalgs
-
Team project name - Digital Workplace
-
Release definition - directory-webapp-release/202001.5
-
Agent - Hosted windows-2019
Issue Description
Task logs
Troubleshooting
Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting
Error logs
2020-01-15T16:49:40.7822699Z ##[debug][Azure Call]Getting resource details for azure storage account resource: dgsappdirectoryint with resource type: Microsoft.Storage/storageAccounts 2020-01-15T16:49:42.1074825Z ##[debug]System.NullReferenceException: Object reference not set to an instance of an object. 2020-01-15T16:49:42.1096413Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount..ctor(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 27 2020-01-15T16:49:42.1111457Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount.Create(StorageAccount storageAccount, IStorageManagementClient client) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 87 2020-01-15T16:49:42.1132108Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.<>c__DisplayClass2.<WriteStorageAccountList>b__1(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161 2020-01-15T16:49:42.1146318Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.GeneralUtilities.ForEach[T](IEnumerable
1 sequence, Action1 action) in d:\workspace\powershell-sign\src\Common\Commands.Common\GeneralUtilities.cs:line 147 2020-01-15T16:49:42.1167097Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.WriteStorageAccountList(IEnumerable
1 storageAccounts) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161
2020-01-15T16:49:42.1180004Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord() in d:\workspace\powershell-sign\src\Common\Commands.Common\AzurePSCmdlet.cs:line 610
2020-01-15T16:49:42.3306724Z ##[debug]Processed: ##vso[task.logissue type=error;code={“Task_Internal_Error”:“TemporaryCopyingToBlobContainerFailed”};]
2020-01-15T16:49:42.3351803Z ##[debug]Trying to disconnect from Azure and clear context at process scope
2020-01-15T16:49:42.5623196Z ##[debug]Caught exception from task script.
2020-01-15T16:49:42.5655643Z ##[debug]Error record:
2020-01-15T16:49:42.6254818Z ##[debug]Get-AzureRmStorageAccount : Object reference not set to an instance of an object.
2020-01-15T16:49:42.6267065Z ##[debug]At D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureUtilityGTE1.0.ps1:27 char:47
2020-01-15T16:49:42.6279224Z ##[debug]+ … ResourceDetails = Get-AzureRmStorageAccount -ErrorAction Stop | Where …
2020-01-15T16:49:42.6290268Z ##[debug]+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-01-15T16:49:42.6303078Z ##[debug] + CategoryInfo : CloseError: (😃 [Get-AzureRmStorageAccount], NullReferenceException
2020-01-15T16:49:42.6314916Z ##[debug] + FullyQualifiedErrorId : Microsoft.Azure.Commands.Management.Storage.GetAzureStorageAccountCommand
2020-01-15T16:49:42.6326337Z ##[debug]
2020-01-15T16:49:42.6345394Z ##[debug]Script stack trace:
2020-01-15T16:49:42.6379888Z ##[debug]at Get-AzureStorageAccountResourceGroupName, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureUtilityGTE1.0.ps1: line 27
2020-01-15T16:49:42.6392058Z ##[debug]at Get-AzureStorageKeyFromARM, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureUtilityRest.ps1: line 28
2020-01-15T16:49:42.6406408Z ##[debug]at Get-StorageKey, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\Utility.ps1: line 149
2020-01-15T16:49:42.6422458Z ##[debug]at <ScriptBlock>, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureFileCopy.ps1: line 101
2020-01-15T16:49:42.6434089Z ##[debug]at <ScriptBlock>, <No file>: line 1
2020-01-15T16:49:42.6446172Z ##[debug]at <ScriptBlock>, <No file>: line 22
2020-01-15T16:49:42.6459269Z ##[debug]at <ScriptBlock>, <No file>: line 18
2020-01-15T16:49:42.6470994Z ##[debug]at <ScriptBlock>, <No file>: line 1
2020-01-15T16:49:42.6489785Z ##[debug]Exception:
2020-01-15T16:49:42.6511296Z ##[debug]System.NullReferenceException: Object reference not set to an instance of an object.
2020-01-15T16:49:42.6523029Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount…ctor(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 27
2020-01-15T16:49:42.6535296Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount.Create(StorageAccount storageAccount, IStorageManagementClient client) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 87
2020-01-15T16:49:42.6547081Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.<>c__DisplayClass2.<WriteStorageAccountList>b__1(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161
2020-01-15T16:49:42.6558457Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.GeneralUtilities.ForEach[T](IEnumerable1 sequence, Action
1 action) in d:\workspace\powershell-sign\src\Common\Commands.Common\GeneralUtilities.cs:line 147
2020-01-15T16:49:42.6569978Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.WriteStorageAccountList(IEnumerable1 storageAccounts) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161 2020-01-15T16:49:42.6581642Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord() in d:\workspace\powershell-sign\src\Common\Commands.Common\AzurePSCmdlet.cs:line 610 2020-01-15T16:49:42.6641627Z ##[error]Object reference not set to an instance of an object. 2020-01-15T16:49:42.6642015Z ##[debug]Processed: ##vso[task.logissue type=error]Object reference not set to an instance of an object.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 17 (12 by maintainers)
Yes, this was a breaking change with Azure File Copy V4 (hence a new version).
The Service Principal needs to also have Storage Blob Data Contributor permissions on the Storage account. This is because of AzCopy.exe logging in using the Service Principal Credentials in V4 task. Can you assign the permission and re-queue the pipeline. More details on Azure File Copy V4 task and roles.
AzureFileCopyV4 task is based on new Az modules as opposed to older AzureRm modules. V4 is the way going forward. That being said, we are working on finding the root cause in V2 and V3, and will come with a fix asap.
@trevonmckay We have fixed the issue in latest version of AzureFileCopy. It will take at least 5-6 weeks to get deployed.
@asranja Just to follow up that did fix the issue and the V4 task is now working after I gave the Service Principal the correct permissions on the Storage Account. Thank you!
If you need any tests of the V2/V3 tasks, I would still be happy to test those and verify they will fix the reported issue.
Thank you again!