aiobotocore: 2.4.0: pytest is failing

Describe the bug A clear and concise description of what the bug is.

Checklist

  • I have reproduced in environment where pip check passes without errors
  • I have provided pip freeze results
  • I have provided sample code or detailed way to reproduce
  • I have tried the same code in botocore to ensure this is an aiobotocore specific issue
  • I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue
  • I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection

pip freeze results

Package                       Version
----------------------------- -----------------
aiohttp                       3.8.1
aioitertools                  0.10.0
aiosignal                     1.2.0
alabaster                     0.7.12
async-timeout                 4.0.2
attrs                         22.1.0
Babel                         2.10.2
boto3                         1.24.57
botocore                      1.27.57
Brlapi                        0.8.3
build                         0.8.0
cffi                          1.15.1
charset-normalizer            2.1.0
click                         8.1.2
codespell                     2.1.0
cryptography                  36.0.2
cycler                        0.11.0
dill                          0.3.5.1
distro                        1.7.0
docutils                      0.18.1
extended-project              1.2.3
extras                        1.0.0
fixtures                      4.0.0
Flask                         2.2.1
Flask-Cors                    3.0.10
fonttools                     4.36.0
frozenlist                    1.3.1
gpg                           1.17.1-unknown
idna                          3.3
imagesize                     1.4.1
importlib-metadata            4.12.0
iniconfig                     1.1.1
itsdangerous                  2.1.2
Jinja2                        3.1.1
jmespath                      1.0.0
joblib                        1.1.0
kiwisolver                    1.4.4
libcomps                      0.1.18
louis                         3.22.0
MarkupSafe                    2.1.1
matplotlib                    3.5.3
moto                          3.1.10.dev0
multidict                     6.0.2
numpy                         1.23.1
olefile                       0.46
packaging                     21.3
pbr                           5.9.0
pep517                        0.12.0
Pillow                        9.2.0
pip                           22.2.1
pluggy                        1.0.0
ply                           3.11
Pollster                      0.1.6
py                            1.11.0
pycparser                     2.21
Pygments                      2.13.0
PyGObject                     3.42.2
pyparsing                     3.0.9
pytest                        7.1.2
pytest-asyncio                0.19.0
python-dateutil               2.8.2
pytz                          2022.1
requests                      2.28.1
responses                     0.21.0
rpm                           4.17.0
s3transfer                    0.6.0
scikit-learn                  1.1.2
scipy                         1.9.0
setuptools                    65.3.0
simple-project                1.2.3
six                           1.16.0
sklearn                       0.0
snowballstemmer               2.2.0
Sphinx                        5.1.1
sphinxcontrib-applehelp       1.0.2.dev20220730
sphinxcontrib-devhelp         1.0.2.dev20220730
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1.dev20220730
sphinxcontrib-qthelp          1.0.3.dev20220730
sphinxcontrib-serializinghtml 1.1.5
testtools                     2.5.0
threadpoolctl                 3.1.0
tomli                         2.0.1
typing_extensions             4.2.0
urllib3                       1.26.11
Werkzeug                      2.2.2
wheel                         0.37.1
wrapt                         1.14.0
xmltodict                     0.12.0
yarl                          1.8.1
zipp                          3.8.1

Environment:

  • Python Version: 3.8.13
  • OS name and version: Linux x86/64

Additional context I’m packaging your module as an rpm package so I’m using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

  • python3 -sBm build -w --no-isolation
  • because I’m calling build with --no-isolation I’m using during all processes only locally installed modules
  • install .whl file in </install/prefix>
  • run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Becaus eone unit is failing on missing fixture I’m assuming that at least one module is missing in my build env. May I ask fo rdome hints what may be missing?

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 31 (3 by maintainers)

Most upvoted comments

Tested ant it does not change anything and more even units fails.

I also got many failures from tests/test_basic_s3.py with botocore 1.29.16, so I keep botocore 1.27.96 for now and only one test fails (test_load_sso_credentials_with_cache_expired). As a side note, support for botocore >= 1.28 is tracked in https://github.com/aio-libs/aiobotocore/issues/976.

Tested ant it does not change anything and more even units fails.

========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_basic_s3.py:415: Not supported
ERROR tests/test_basic_s3.py::test_can_get_bucket_location - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_delete_urlencoded_object - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate_with_page_size - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_search_paginate - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate_iterator - assert 200 == 204
ERROR tests/test_basic_s3.py::test_result_key_iters - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_get_and_put_object - assert 200 == 204
ERROR tests/test_basic_s3.py::test_adaptive_retry - assert 200 == 204
ERROR tests/test_basic_s3.py::test_get_object_stream_wrapper - assert 200 == 204
ERROR tests/test_basic_s3.py::test_get_object_stream_context - assert 200 == 204
ERROR tests/test_basic_s3.py::test_paginate_max_items - botocore.exceptions.ParamValidationError: Parameter validation failed:
ERROR tests/test_basic_s3.py::test_paginate_within_page_boundaries - assert 200 == 204
ERROR tests/test_basic_s3.py::test_unicode_key_put_list[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateBucket op...
ERROR tests/test_basic_s3.py::test_unicode_key_put_list[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObjectVersi...
ERROR tests/test_basic_s3.py::test_unicode_system_character[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateBucke...
ERROR tests/test_basic_s3.py::test_unicode_system_character[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObjectV...
ERROR tests/test_basic_s3.py::test_non_normalized_key_paths - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_quoted_char - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_query_string - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_copy_with_dict_form - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_copy_with_dict_form_with_version - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_s3_metadata - assert 200 == 204
ERROR tests/test_basic_s3.py::test_presign_with_existing_query_string_values[False-s3-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId...
ERROR tests/test_basic_s3.py::test_presign_with_existing_query_string_values[False-s3-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId...
ERROR tests/test_basic_s3.py::test_presign_sigv4[False-s3v4-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateB...
ERROR tests/test_basic_s3.py::test_presign_sigv4[False-s3v4-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObj...
ERROR tests/test_basic_s3.py::test_can_follow_signed_url_redirect[False-s3v4] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ...
ERROR tests/test_basic_s3.py::test_can_follow_signed_url_redirect[False-s3v4] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ...
ERROR tests/test_basic_s3.py::test_bucket_redirect[False-us-west-2-eu-west-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ...
ERROR tests/test_basic_s3.py::test_head_object_keys[s3v4] - assert 200 == 204
ERROR tests/boto_tests/test_credentials.py::test_load_sso_credentials_without_cache
FAILED tests/test_basic_s3.py::test_succeed_proxy_request[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListBuckets o...
FAILED tests/test_basic_s3.py::test_can_delete_urlencoded_object - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operat...
FAILED tests/test_basic_s3.py::test_can_paginate - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retries: 4)...
FAILED tests/test_basic_s3.py::test_can_paginate_with_page_size - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ...
FAILED tests/test_basic_s3.py::test_can_search_paginate - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retr...
FAILED tests/test_basic_s3.py::test_can_paginate_iterator - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max re...
FAILED tests/test_basic_s3.py::test_result_key_iters - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: The spe...
FAILED tests/test_basic_s3.py::test_can_get_and_put_object - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max r...
FAILED tests/test_basic_s3.py::test_adaptive_retry - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retries: ...
FAILED tests/test_basic_s3.py::test_get_object_stream_wrapper - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ma...
FAILED tests/test_basic_s3.py::test_get_object_stream_context - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ma...
FAILED tests/test_basic_s3.py::test_paginate_max_items - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the CreateMultipartUpload oper...
FAILED tests/test_basic_s3.py::test_paginate_within_page_boundaries - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reac...
FAILED tests/test_basic_s3.py::test_non_normalized_key_paths - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation:...
FAILED tests/test_basic_s3.py::test_copy_with_quoted_char - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: Th...
FAILED tests/test_basic_s3.py::test_copy_with_query_string - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: T...
FAILED tests/test_basic_s3.py::test_can_copy_with_dict_form - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: ...
FAILED tests/test_basic_s3.py::test_can_copy_with_dict_form_with_version - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObjec...
FAILED tests/test_basic_s3.py::test_copy_with_s3_metadata - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max re...
FAILED tests/test_basic_s3.py::test_bucket_redirect[False-us-west-2-eu-west-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the...
FAILED tests/test_basic_s3.py::test_head_object_keys[s3v4] - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max r...
FAILED tests/test_lambda.py::test_run_lambda - botocore.exceptions.ClientError: An error occurred (500) when calling the CreateFunction operation (reached max retries: 4...
FAILED tests/test_mturk.py::test_mturk_stubber - AttributeError: 'MTurk' object has no attribute 'list_hi_ts_for_qualification_type'
FAILED tests/test_patches.py::test_patches - assert False
FAILED tests/test_waiter.py::test_sqs - botocore.parsers.ResponseParserError: Unable to parse response (syntax error: line 1, column 0), invalid XML received. Further re...
FAILED tests/boto_tests/test_credentials.py::test_get_credentials - botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://[fd00:ec2...
FAILED tests/boto_tests/test_credentials.py::test_load_sso_credentials_with_cache_expired - botocore.exceptions.StubAssertionError: Error getting response stub for opera...
FAILED tests/python3.8/test_eventstreams.py::test_kinesis_stream_json_parser - botocore.exceptions.NoRegionError: You must specify a region.
FAILED tests/python3.8/boto_tests/test_signers.py::test_signers_generate_presigned_urls - AssertionError: expected call not found.
============================================== 29 failed, 146 passed, 1 skipped, 47 warnings, 32 errors in 209.63s (0:03:29) ===============================================

Looks like test suite needs access to AWS services

docker is used by moto for lambda testing