azure-storage-azcopy: Copy fails (CannotVerifyCopySource) when storage account firewall is enabled
Using azcopy 10.3.2 (Windows), with a storage account firewall enabled on the source and destination, copy (or sync) between file shares fails with CannotVerifyCopySource.
azcopy copy https://nicktestsource.file.core.windows.net/source/test.txt?sv=2018-11-09&sig=<REDACTED>=2019-11-21T10%3A07%3A46Z&se=2020-11-21T10%3A07%3A46Z&srt=sco&ss=f&sp=racupwdl https://nicktestdest.file.core.windows.net/dest/test.txt?sv=2018-11-09&sig=<REDACTED>&st=2019-11-21T10%3A01%3A58Z&se=2020-11-21T10%3A01%3A58Z&srt=sco&ss=f&sp=racupwdl
Copy fails with:
INFO: Authentication failed, it is either not correct, or expired, or does not have the correct permission -> github.com/Azure/azure-storage-file-go/azfile.newStorageError, /home/vsts/go/pkg/mod/github.com/!azure/azure-storage-file-go@v0.6.0/azfile/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=CannotVerifyCopySource) =====
Description=This request is not authorized to perform this operation.
RequestId:b50a48eb-701a-0026-2f53-a0d39d000000
Time:2019-11-21T10:07:49.6641367Z, Details:
Code: CannotVerifyCopySource
PUT https://nicktestdest.file.core.windows.net/dest/test.txt?comp=range&se=2020-11-21t10%3A07%3A46z&sig=-REDACTED-&sp=racupwdl&srt=sco&ss=f&st=2019-11-21t10%3A07%3A46z&sv=2018-11-09&timeout=901
Content-Length: [0]
User-Agent: [AzCopy/10.3.2 Azure-Storage/0.6.0 (go1.13; Windows_NT)]
X-Ms-Client-Request-Id: [3f5dbe62-5b0a-4528-6a02-3b891c5f387b]
X-Ms-Copy-Source: [https://nicktestsource.file.core.windows.net/source/test.txt?se=2020-11-21t10%3A07%3A46z&sig=-REDACTED-&sp=racupwdl&srt=sco&ss=f&st=2019-11-21t10%3A07%3A46z&sv=2018-11-09]
X-Ms-Range: [bytes=0-4]
X-Ms-Source-Range: [bytes=0-4]
X-Ms-Version: [2019-02-02]
X-Ms-Write: [update]
--------------------------------------------------------------------------------
RESPONSE Status: 403 This request is not authorized to perform this operation.
Content-Length: [248]
Content-Type: [application/xml]
Date: [Thu, 21 Nov 2019 10:07:48 GMT]
Server: [Windows-Azure-File/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Client-Request-Id: [3f5dbe62-5b0a-4528-6a02-3b891c5f387b]
X-Ms-Error-Code: [CannotVerifyCopySource]
X-Ms-Request-Id: [b50a48eb-701a-0026-2f53-a0d39d000000]
X-Ms-Version: [2019-02-02]
Problem can be replicated by attempting a copy between shares on different storage accounts which both have a storage account firewall configured.
Disabling the storage account firewall on both the source and destination works.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 3
- Comments: 28 (11 by maintainers)
Why is this ticket closed? This issue and #1673 are not fixed.
Should be fixed by now: https://azure.microsoft.com/en-us/updates/copy-blob-support-over-private-endpoints/
I think this issue should be reopened. I don’t want to create yet another duplicate issue as this and #1673 already exist. I have created a “project” that can be quickly deployed to Azure to reproduce this problem: https://github.com/tompaana/blob-copy-test
Update: @derdanu helped me to find a workaround for the time being: Add IP ranges of the storages in the destination location to the source storage firewall rules to allow access. See https://github.com/tompaana/blob-copy-test#observed-copy-results
@sanmai-NL I moved to a different team a few years ago. @adreed-msft, can you take a look at the comments above and reply here please?
Hi John, I’m interested in any workarounds you might find!
We see the same issue in private networking setup (no public IP’s to whitelist). AzCopy fails with 403 where the VM, sitting in a VNET, can access both storage accounts.
This is outstandingly poor IMHO … copy from a to b in the cloud should be as simple as an old dos style copy folder a to folder b … Private link endpoints have given no end of issues not a well thought out design more of an after thought bolt on… I’ve spent a few weeks getting all our on prem and azure firewalls opened up to allow a copy from one SA to another cloud side and this still seems to be an issue for over 3 YEARS after it was first reported. … #PoorCustomerService
@adreed-msft perhaps you can help ?
Still experiencing this issue when trying to transfer from blob container to blob container in two separate private endpoints.
Using a computer that can access both the source and destination (e.g. I can use
azcopy list ...to both containers), I fail when trying to copy from source to destination with an authorization failure. Appears to still be an issue with the destination storage account sending the PUT request?Update: there isn’t any short-term workaround in V10. I’m re-opening this issue, to remind us to post an update here when the Service-side issue is resolved. When that is resolved, v10 will work and we can close this issue.
The config change is not yet available. I’m investigating other workarounds. Please reply here if you need details.
It’s only impossible in the short term. A change is currently rolling out to the Azure Files service. Once that change is rolled out, it should work like this:
I don’t know exactly when this change will finish rolling out. If you have a particularly urgent need, please email me and I’ll make some inquiries for you. My email address is in my profile page.
cc @adreed-msft This config change is good news, and will resolve the issue you mentioned above with no effort from users.