azure-cli: `az storage blob download-batch --pattern` fails while individual blobs can be downloaded

Describe the bug

When trying to batch download a few blob using az storage blob download-batch with a --pattern the cli command takes too long (more than 10 minutes a for a few small blobs).

az storage blob download-batch -d . -s '$web'  --pattern 'run-5123825179-d6e24fafe89ddf4293bb47ad33472639689be181-test_linux/*.jsonl' --connection-string  "$AZURE_CONN"

We run the command periodically on GitHub Actions, here is a sample run: https://github.com/microsoft/playwright/actions/runs/5123825179/jobs/9215588223

Related command

az storage blob download-batch --pattern 'foo/*.json'

Errors

The command takes too long and after 10 minutes just gets killed by a watchdog process.

Issue script & Debug output

Command:

az storage blob download-batch -d . -s '$web'  --pattern 'run-5123825179-d6e24fafe89ddf4293bb47ad33472639689be181-test_linux/*.jsonl' --connection-string  "$AZURE_CONN" --debug 2>&1 | tee log

Output:

DEBUG: cli.knack.cli: Command arguments: ['storage', 'blob', 'download-batch', '-d', '.', '-s', '$web', '--pattern', 'run-5123825179-d6e24fafe89ddf4293bb47ad33472639689be181-test_linux/*.jsonl', '--connection-string', 'DefaultEndpointsProtocol=https;AccountName=mspwblobreport;AccountKey=HP+q0_<redacted>_iNg==;EndpointSuffix=core.windows.net', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x104e05e10>, <function OutputProducer.on_global_arguments at 0x104e92b90>, <function CLIQuery.on_global_arguments at 0x104f0c040>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'storage': ['azure.cli.command_modules.storage']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: storage                   0.028        58       272
DEBUG: cli.azure.cli.core: Total (1)                 0.028        58       272
DEBUG: cli.azure.cli.core: Loaded 58 groups, 272 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : storage blob download-batch
DEBUG: cli.azure.cli.core: Command table: storage blob download
DEBUG: cli.azure.cli.core: remaining    :                       batch
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x10591de10>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/Users/yurys/.azure/commands/2023-05-30.11-34-29.storage_blob_download-batch.16601.log'.
INFO: az_command_data_logger: command args: storage blob download-batch -d {} -s {} --pattern {} --connection-string {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x10592e7a0>]
DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
  File "/opt/homebrew/Cellar/azure-cli/2.49.0/libexec/lib/python3.10/site-packages/azure/cli/core/profiles/_shared.py", line 655, in _get_attr
    op = getattr(op, part)
AttributeError: module 'azure.mgmt.storage.v2022_09_01.models' has no attribute 'ActiveDirectoryPropertiesAccountType'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
  File "/opt/homebrew/Cellar/azure-cli/2.49.0/libexec/lib/python3.10/site-packages/azure/cli/core/profiles/_shared.py", line 655, in _get_attr
    op = getattr(op, part)
AttributeError: module 'azure.mgmt.storage.v2022_09_01.models' has no attribute 'ListKeyExpand'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
  File "/opt/homebrew/Cellar/azure-cli/2.49.0/libexec/lib/python3.10/site-packages/azure/cli/core/profiles/_shared.py", line 655, in _get_attr
    op = getattr(op, part)
AttributeError: module 'azure.mgmt.storage.v2022_09_01.models' has no attribute 'CorsRuleAllowedMethodsItem'

DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x105b484c0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x105b485e0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x104e92c20>, <function CLIQuery.handle_query_parameter at 0x104f0c0d0>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x105b48550>]
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage auth_mode value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account_url value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage sas_token value from environment variables or config file.
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): mspwblobreport.blob.core.windows.net:443
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYloyY3JGajQvUndWUVl1UUVZc3dFdmJYTndkMkpzYjJKeVpYQnZjblFCTURGRU9UZzRORU16TWtFeE5qTTNRUzhrZDJWaUFUQXhSRGs0T0RSRU5UTXdNakV3T0RBdmNuVnVMVFV3TWpnd05qazFORFl0WXpoaE5ESmhaakV4WmpkbU5tSTROR1ZqT0dJeU5tSmtObVE1WTJZd1pHSXhZalF3TkRZeVpDMTBjbUZqYVc1blgyeHBiblY0TDNKbGMyOTFjbU5sY3k4MU9USTFNR1JqWW1FeVpqRTBOVFpsT0dReVpETTROMlZtTkRNMk5HSTVPV016WmpNelkyVTRMbnBwY0JZQUFBQT0hMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzQ6MzMuMzk3MjE0OVoh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYVg2Wk9mcTlXUzN0b0JHTGtCR0xNQkwyMXpjSGRpYkc5aWNtVndiM0owQVRBeFJEazRPRFJETXpKQk1UWXpOMEV2SkhkbFlnRXdNVVE1T0RnMFJEVXpNREl4TURnd0wzSjFiaTAxTURJNE1EWTVOVFEyTFdNNFlUUXlZV1l4TVdZM1pqWmlPRFJsWXpoaU1qWmlaRFprT1dObU1HUmlNV0kwTURRMk1tUXRkSEpoWTJsdVoxOXNhVzUxZUM5eVpYTnZkWEpqWlhNdll6SmpZekl4TURJM1lXUXhaV0V6TkRObE4yTTNNbU0xTldJMk1URTNZVEJsTUdJMVpHUmlNeTU2YVhBV0FBQUEhMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzQ6MzguMDUyMzYxNloh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYlQzK1NlaGE3WjFRa1l1UUVZc3dFdmJYTndkMkpzYjJKeVpYQnZjblFCTURGRU9UZzRORU16TWtFeE5qTTNRUzhrZDJWaUFUQXhSRGs0T0RSRU5UTXdNakV3T0RBdmNuVnVMVFV3TWpnd09UWTJNVGN0T0dabFpURTNOVGc1Tm1NMU9HWmtaV1F4WmpJeU1ESmhZV1JqWmpVNU1UZ3hNVEl5WWpsbE5DMTBjbUZqYVc1blgyeHBiblY0TDNKbGMyOTFjbU5sY3k4eU56WXdOVFl5TlRReVpEaGlNamN4TXpNMFlUZ3hOR0psWm1RM056WXlaR0kyWkdKaFkyWTJMbnBwY0JZQUFBQT0hMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzQ6NDEuNzE3MTE0NVoh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYjN6TURVM3FYWm1Tc1l1UUVZc3dFdmJYTndkMkpzYjJKeVpYQnZjblFCTURGRU9UZzRORU16TWtFeE5qTTNRUzhrZDJWaUFUQXhSRGs0T0RSRU5UTXdNakV3T0RBdmNuVnVMVFV3TWpnd09UWTJNVGN0T0dabFpURTNOVGc1Tm1NMU9HWmtaV1F4WmpJeU1ESmhZV1JqWmpVNU1UZ3hNVEl5WWpsbE5DMTBjbUZqYVc1blgyeHBiblY0TDNKbGMyOTFjbU5sY3k4NU1ETTRPV0psT0RSaE9UVTFaalZpTnpRMk1qTXdaRGs1WXprMk5UTTRaVEE1TjJZMVpXWTRMbnBwY0JZQUFBQT0hMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzQ6NDYuMjAxMzY1Nloh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYTFpdEdYdXZ1ancrd0JHTGtCR0xNQkwyMXpjSGRpYkc5aWNtVndiM0owQVRBeFJEazRPRFJETXpKQk1UWXpOMEV2SkhkbFlnRXdNVVE1T0RnMFJEVXpNREl4TURnd0wzSjFiaTAxTURJNE1EazJOakUzTFRobVpXVXhOelU0T1Raak5UaG1aR1ZrTVdZeU1qQXlZV0ZrWTJZMU9URTRNVEV5TW1JNVpUUXRkSEpoWTJsdVoxOXNhVzUxZUM5eVpYTnZkWEpqWlhNdlpqWmpZbU16T0ROa09EVXhZemsyTmpjeFlUZzJPR1EzWXpNd01XUmhZMkV4WW1FNE5qQmpNaTU2YVhBV0FBQUEhMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzQ6NDkuOTU5MDYzM1oh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYS9sZEx3K3RqcC80c0JHTGtCR0xNQkwyMXpjSGRpYkc5aWNtVndiM0owQVRBeFJEazRPRFJETXpKQk1UWXpOMEV2SkhkbFlnRXdNVVE1T0RnMFJEVXpNREl4TURnd0wzSjFiaTAxTURJNE5URTRNVGczTFdKa016TmxOV1k0WXpBeU5qUXpNR1F6WlRVNE1UY3pPVEF5TWpZM1lUazVPVEl6WXpoaE1qSXRkSEpoWTJsdVoxOXNhVzUxZUM5eVpYTnZkWEpqWlhNdk5XUTFaVE5oT1RjNE16VmhNak16WVRBd1lURTBaalU1WXprM05UTmxOR0kzTkRnNU56VTBZUzU2YVhBV0FBQUEhMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzQ6NTQuNDU2MzA2MFoh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYlB3WVMwbFpXYW9lSUJHTGtCR0xNQkwyMXpjSGRpYkc5aWNtVndiM0owQVRBeFJEazRPRFJETXpKQk1UWXpOMEV2SkhkbFlnRXdNVVE1T0RnMFJEVXpNREl4TURnd0wzSjFiaTAxTURJNE5URTRNVGczTFdKa016TmxOV1k0WXpBeU5qUXpNR1F6WlRVNE1UY3pPVEF5TWpZM1lUazVPVEl6WXpoaE1qSXRkSEpoWTJsdVoxOXNhVzUxZUM5eVpYTnZkWEpqWlhNdll6VTJaR00yWVdFMU56SXdNalV6WVRKaVlqbGlNamN5TVdVMVlUQTROVFEwWmpjMk5qSTFNUzU2YVhBV0FBQUEhMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzQ6NTguODY2NjAyOVoh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYVQzcTdYeS9UODltWVl1UUVZc3dFdmJYTndkMkpzYjJKeVpYQnZjblFCTURGRU9UZzRORU16TWtFeE5qTTNRUzhrZDJWaUFUQXhSRGs0T0RSRU5UTXdNakV3T0RBdmNuVnVMVFV3TWpnM016Y3hNalV0WW1Rd09UQmxOamRrWm1RellUUmhPRGRrWWpRNU9HRTNPV05tWVRBeFpqVXhNRGd5TVRKaVl5MTBjbUZqYVc1blgyeHBiblY0TDNKbGMyOTFjbU5sY3k4eVl6TTNOVGN4WmpWaU5EazFORFkwTWpSa056RXhPV1l5TWpBMk1XTTBNalJsT0RrNFpXUmlMbnBwY0JZQUFBQT0hMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzU6MDIuNjE2MzA0NVoh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYWRvOHpsN3Z1czRuWVl1UUVZc3dFdmJYTndkMkpzYjJKeVpYQnZjblFCTURGRU9UZzRORU16TWtFeE5qTTNRUzhrZDJWaUFUQXhSRGs0T0RSRU5UTXdNakV3T0RBdmNuVnVMVFV3TWpnM016Y3hNalV0WW1Rd09UQmxOamRrWm1RellUUmhPRGRrWWpRNU9HRTNPV05tWVRBeFpqVXhNRGd5TVRKaVl5MTBjbUZqYVc1blgyeHBiblY0TDNKbGMyOTFjbU5sY3k4NU5ESmlaV1JrWlRSaVkyUm1NR1UyTlRkak5HWXpZelZoT0dJNU1HRm1aREkxTUdJM09UY3dMbnBwY0JZQUFBQT0hMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzU6MDcuMjkyNDM4MFoh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYjF2b082enVERjZaQUJHTGtCR0xNQkwyMXpjSGRpYkc5aWNtVndiM0owQVRBeFJEazRPRFJETXpKQk1UWXpOMEV2SkhkbFlnRXdNVVE1T0RnMFJEVXpNREl4TURnd0wzSjFiaTAxTURJNE56TTNNVEkxTFdKa01Ea3daVFkzWkdaa00yRTBZVGczWkdJME9UaGhOemxqWm1Fd01XWTFNVEE0TWpFeVltTXRkSEpoWTJsdVoxOXNhVzUxZUM5eVpYTnZkWEpqWlhNdlptRXdaV1k0TXpZNVpEVXlPV1ZrWm1NNU9HRTRaVFE0TjJJek5EVm1Zek5oTkRFek5qQTBOaTU2YVhBV0FBQUEhMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzU6MTEuMDUzMTMzNloh HTTP/1.1" 200 None
DEBUG: urllib3.connectionpool: https://mspwblobreport.blob.core.windows.net:443 "GET /%24web?restype=container&comp=list&marker=2%21416%21MDAwMjY4IVZCYU8wNGlKeWVUM294RVl1UUVZc3dFdmJYTndkMkpzYjJKeVpYQnZjblFCTURGRU9UZzRORU16TWtFeE5qTTNRUzhrZDJWaUFUQXhSRGs0T0RSRU5UTXdNakV3T0RBdmNuVnVMVFV3TWpnM05EQTVPRGd0TUdVeFlXVmhZV1ZqWmpGaU1HTmxNR1V3WkRka1pHVXhaRFV4TXpnellqZzJZelJqTTJZM01pMTBjbUZqYVc1blgyeHBiblY0TDNKbGMyOTFjbU5sY3k4MVpqa3lNREF6WTJFM1l6bGxOVFl6TXpNeE9HRmlNak00Wmpnd05ERTNaVEF6TmpnM05EWmpMbnBwY0JZQUFBQT0hMDAwMDI4ITIwMjMtMDUtMzBUMTg6MzU6MTUuNDAxNDY4NFoh HTTP/1.1" 200 None

Expected behavior

There is less than 10 blobs 35Mb each which are expected to be downloaded in less than a minute:

Environment Summary

azure-cli 2.49.0

core 2.49.0 telemetry 1.0.8

Dependencies: msal 1.20.0 azure-mgmt-resource 22.0.0

Python location ‘/opt/homebrew/Cellar/azure-cli/2.49.0/libexec/bin/python’ Extensions directory ‘/Users/yurys/.azure/cliextensions’

Python (Darwin) 3.10.11 (main, Apr 7 2023, 07:24:53) [Clang 14.0.0 (clang-1400.0.29.202)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response