azure-cli: Cannot delete full directory structure from Directory in File Share

Is your feature request related to a problem? Please describe.

The az storage file upload-batch command can be used to upload a full directory structure to an Azure File Share or Directory.

The az storage file delete-batch command can be used to recursively delete all the files in an Azure File Share or Directory. It does however leave all the empty subfolders.

There does not seem to be a way of deleting all the empty subfolders - unless each subfolder is deleted individually. This requires prior knowledge of the folder names.

Describe the solution you’d like

The az storage file delete-batch command should have an option to delete empty subfolders. Alternatively, or additionally, the az storage directory delete should have an option to delete all subfolders.

Describe alternatives you’ve considered

Note that is is possible to delete the whole file share using az storage share delete --name sparkysfileshare just not a specific directory within a file share.

Additional context

Here’s an example workflow:

> az storage directory create --name sparkysfolder --share-name sparkysfileshare 
{
  "created": true
}

> az storage file upload-batch --destination sparkysfileshare --source I:\temp\test --destination-path sparkysfolder --pattern * 

uploading I:\temp\test\test1.txt
Finished[#############################################################]  100.0000%
uploading I:\temp\test\subfolder\test2.txt
Finished[#############################################################]  100.0000%
[
  "https://vsoftfilestorageac.file.core.windows.net/sparkysfileshare/sparkysfolder/test1.txt",
  "https://vsoftfilestorageac.file.core.windows.net/sparkysfileshare/sparkysfolder/subfolder/test2.txt"
]

> az storage file list --share-name sparkysfileshare --path sparkysfolder

[
  {
    "metadata": null,
    "name": "test1.txt",
    "properties": {
      "contentLength": 4,
      "contentRange": null,
      "contentSettings": {
        "cacheControl": null,
        "contentDisposition": null,
        "contentEncoding": null,
        "contentLanguage": null,
        "contentMd5": null,
        "contentType": null
      },
      "copy": {
        "completionTime": null,
        "id": null,
        "progress": null,
        "source": null,
        "status": null,
        "statusDescription": null
      },
      "etag": null,
      "lastModified": null,
      "serverEncrypted": null
    },
    "type": "file"
  },
  {
    "metadata": null,
    "name": "subfolder",
    "properties": {
      "etag": null,
      "lastModified": null,
      "serverEncrypted": null
    },
    "type": "dir"
  }
]

> az storage file delete-batch --source sparkysfileshare --pattern sparkysfolder\* --account-name vsoftfilestorageac 

> az storage file list --share-name sparkysfileshare --path sparkysfolder

[
  {
    "metadata": null,
    "name": "subfolder",
    "properties": {
      "etag": null,
      "lastModified": null,
      "serverEncrypted": null
    },
    "type": "dir"
  }
]

> az storage directory delete --name sparkysfolder --share-name sparkysfileshare 

The command failed with an unexpected error. Here is the traceback:

The specified directory is not empty. ErrorCode: DirectoryNotEmpty
<?xml version="1.0" encoding="utf-8"?><Error><Code>DirectoryNotEmpty</Code><Message>The specified directory is not empty.
RequestId:17d1cf8b-001a-000b-7fab-f584e4000000
Time:2019-04-18T05:59:17.7350008Z</Message></Error>
Traceback (most recent call last):
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\knack\knack\cli.py", line 206, in invoke
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-cli-core\azure\cli\core\commands\__init__.py", line 326, in execute
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-cli-core\azure\cli\core\commands\__init__.py", line 384, in _run_jobs_serially
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-cli-core\azure\cli\core\commands\__init__.py", line 377, in _run_job
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\six\six.py", line 693, in reraise
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-cli-core\azure\cli\core\commands\__init__.py", line 354, in _run_job
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-cli-core\azure\cli\core\commands\__init__.py", line 145, in __call__
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-cli-core\azure\cli\core\__init__.py", line 451, in default_command_handler
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-multiapi-storage\azure\multiapi\storage\v2018_03_28\file\fileservice.py", line 1035, in delete_directory
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-multiapi-storage\azure\multiapi\storage\v2018_03_28\common\_error.py", line 97, in _dont_fail_not_exist
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-multiapi-storage\azure\multiapi\storage\v2018_03_28\file\fileservice.py", line 1032, in delete_directory
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-multiapi-storage\azure\multiapi\storage\v2018_03_28\common\storageclient.py", line 381, in _perform_request
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-multiapi-storage\azure\multiapi\storage\v2018_03_28\common\storageclient.py", line 306, in _perform_request
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-multiapi-storage\azure\multiapi\storage\v2018_03_28\common\storageclient.py", line 292, in _perform_request
  File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-z6u9vbmn\azure-multiapi-storage\azure\multiapi\storage\v2018_03_28\common\_error.py", line 115, in _http_error_handler
azure.common.AzureConflictHttpError: The specified directory is not empty. ErrorCode: DirectoryNotEmpty
<?xml version="1.0" encoding="utf-8"?><Error><Code>DirectoryNotEmpty</Code><Message>The specified directory is not empty.
RequestId:17d1cf8b-001a-000b-7fab-f584e4000000
Time:2019-04-18T05:59:17.7350008Z</Message></Error>

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 1
Comments: 36 (6 by maintainers)

Most upvoted comments

you gotta be kidding me…

So i have a tree with 40.000+ folders and over 3.000.000 files, and those need to be deleted on a per item basis ?

+26

ekelmans on Dec 4, 2019

We will add the option to ‘az storage directory delete’ to delete all empty subfolders.

limingu on Apr 22, 2019

`az storage remove`

Remove an entire directory:

az storage remove --account-name "${ASA_NAME}"  --share-name "${SHARE_NAME}" --path abc --recursive

where abc is empty or non-empty directory on Azure File Share.

Remove everything inside Azure Storage File Share:

az storage remove --account-name "${ASA_NAME}"  --share-name "${SHARE_NAME}" --recursive

mloskot on Jun 15, 2023

For those who want to recursively delete a directory on a file share using az CLI rather than lots of powershell, here’s an example script. (it’s still powershell, but easily translates to bash or any other shell):

# recursively delete directory "a/b/c" on fileshare "myfileshare" on storage account "mystorageacc"

$storageAccountName = "mystorageacc"
$fileShareName = "myfileshare"
$directoryToDelete = "a/b/c"
$containerResourceGroup = "myResourceGroup"
 
$mountDirectory = "/mnt/fileshare"

$storageKey = (az storage account keys list --account-name "$storageAccountName" --query "[0].value" --output tsv)

az container create `
  --name 'cleanup' `
  --image 'mcr.microsoft.com/azure-cli:latest' `
  --command-line "rm -r '$mountDirectory/$directoryToDelete'" `
  --restart-policy 'Never' `
  --resource-group $containerResourceGroup `
  --azure-file-volume-account-name $storageAccountName `
  --azure-file-volume-account-key $storageKey `
  --azure-file-volume-share-name $fileShareName `
  --azure-file-volume-mount-path $mountDirectory

Here we mount the file share on an ACI container and remove the directory using the plain rm -r.

Using the image mcr.microsoft.com/azure-cli:latest is not essential, any image with rm will do the trick.

This approach is also immune to the injections or mistakes with the special symbols in directory path such as []?*., unlike the --pattern argument in az storage file delete-batch.

gukoff on Mar 24, 2023

Here’s my contribution. A PowerShell Core script using Az.Storage that deletes File Share folders and files older than a specified date, recursively.

Make sure you’ve logged in (Connect-AzAccount) first. You can use this with Managed Identity on a VM by embedding the Connect-AzAccount call in the script, specifying parameters for -SubscriptionId -Identity and -AccountId (use the Managed Identity’s ClientId GUID).

You can enhance with ForEach-Object -ThrottleLimit # -Parallel but you have to be careful to handle out-of-scope variables or functions.

param(
    [string] $DirectoryPath,
    [string] $StorageAccountName,
    [string] $FileShareName,
    [string] $ResourceGroupName,
    [int] $DaysOld
)

Set-StrictMode -Version Latest

$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"

# Get the current date
$CurrentDate = Get-Date

# Get the date to delete files older than
$DateToDelete = $CurrentDate.AddDays(-$DaysOld)

$StorageAccount = Get-AzStorageAccount -Name $StorageAccountName -ResourceGroupName $ResourceGroupName
# Create a storage context
$StorageContext = $StorageAccount.Context

# Define a function to recursively delete files in a directory
function Remove-FilesRecursively {
    # Get the directory path from the first argument
    $DirectoryPath = $Args[0]
    Write-Output "Entering $DirectoryPath"

    # Get all the files and directories in the directory
    Get-AzStorageFile -ShareName $FileShareName -Path $DirectoryPath -Context $StorageContext | Get-AzStorageFile `
    | ForEach-Object {
        $Item = $_
        $Item.Name
        # If the item is a file, check if it is older than the date to delete and remove it if true
        if ($Item.GetType().Name -eq "AzureStorageFile" -and $Item.LastModified -lt $DateToDelete) {
            Write-Output "Removing $($Item.Name)"
            $Item | Remove-AzStorageFile
        }
        # If the item is a directory, call this function recursively with its path as an argument
        elseif ($Item.GetType().Name -eq "AzureStorageFileDirectory") {
            Write-Output "Going into /$($DirectoryPath)/$($Item.Name)"
            Remove-FilesRecursively "$($DirectoryPath)/$($Item.Name)" @PSBoundParameters
        }
    }

    # Check if the directory is empty after deleting the files and directories inside it
    $folder = Get-AzStorageFile -ShareName $FileShareName -Path $DirectoryPath -Context $StorageContext
    $Contents = $folder | Get-AzStorageFile
    if ($null -eq $Contents) {
        Write-output "Removing empty folder $DirectoryPath"
        $folder | Remove-AzStorageDirectory
    }
}

# Call the function with an empty string as an argument to start from the root directory of the file share
Remove-FilesRecursively $DirectoryPath @PSBoundParameters

silverl on Mar 17, 2023

looks like this is supported now 🎉 I tried it from Microsoft Azure Storage Explorer and it successfully deleted all empty folders recursively. If you want to do it from the command-line with azcopy, this is the command Azure Storage Explorer used:

azcopy remove "https://<account>.file.core.windows.net/<fileshare>/<directory>?<sas-string>" --from-to=FileTrash --recursive

bpossolo on Jun 19, 2022

The command : $d = Get-AzStorageFile -Share $S -ErrorAction SilentlyContinue|select Name does not work with powershell 7 It answers :

Get-AzStorageFile: Cannot bind parameter 'Share'. Cannot convert the "Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageFileShare" value of type "Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageFileShare" to type "Microsoft.Azure.Storage.File.CloudFileShare"

The property $S.CloudFileShare is a Microsoft.Azure.Storage.File.CloudFileShare. Have you tried just updating the call to be $d = Get-AzStorageFile -Share $S.CloudFileShare -ErrorAction SilentlyContinue|select Name?

This solved the issue for me.

joelmforsyth on Jan 18, 2022