airflow: Latest Botocore breaks SQS tests

Body

Our tests are broken in main due to latest botocore failing SQS tests.

Example here: https://github.com/apache/airflow/actions/runs/4887737387/jobs/8724954226

E           botocore.exceptions.ClientError: An error occurred (400) when calling the SendMessage operation: 
<ErrorResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><Error><Type>Sender</Type>
<Code>AWS.SimpleQueueService.NonExistentQueue</Code><Message>The specified queue does not exist for this wsdl version.</Message><Detail/></Error>
<RequestId>ETDUP0OoJOXmn0WS6yWmB0dOhgYtpdVJCVwFWA28lYLKLmGJLAGu</RequestId></ErrorResponse>

The problem seems to come from botocore not recognizing just added queue:

QUEUE_NAME = "test-queue"
QUEUE_URL = f"https://{QUEUE_NAME}"

Even if we replace it with the full queue name that gets returned by the “create_queue” API call to moto, it still does not work with latest botocore:

QUEUE_URL = f"https://sqs.us-east-1.amazonaws.com/123456789012/{QUEUE_NAME}"

Which indicates this likely a real botocore issue.

How to reproduce:

1. Get working venv with [amazon] extra of Airlow (or breeze). Should be (when constraints from main are used):

root@b0c430d9a328:/opt/airflow# pip list  | grep botocore 
aiobotocore                            2.5.0
botocore                               1.29.76

1. pip uninstall aiobotocore
2. pip install --upgrade botocore

root@b0c430d9a328:/opt/airflow# pip list  | grep botocore 
botocore                               1.29.127

1. pytest tests/providers/amazon/aws/sensors/test_sqs.py

Result:

===== 4 failed, 7 passed in 2.43s === 

---

Comparing it to “success case”:

When you run it breeze (with the current constrained botocore):

root@b0c430d9a328:/opt/airflow# pip list  | grep botocore 
aiobotocore                            2.5.0
botocore                               1.29.76

1, pytest tests/providers/amazon/aws/sensors/test_sqs.py

Result:

============ 11 passed in 4.57s =======

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.