moto: moto fails to mock newer boto resulting in AWS being accessed even if boto3.setup_default_session used

This is a consequence of my comment to #2076. Here is the minimal sample code to demonstrate that moto may not mocking everything that is necessary to stop boto from reaching AWS.

s3_test.py:

from moto import mock_s3
import boto3
boto3.setup_default_session() # doesn't work

import unittest

@mock_s3
class SimpleTest(unittest.TestCase):

    def setUp(self):
        boto3.setup_default_session()   # doesn't work either
        self.s3 = boto3.resource('s3', region_name='us-east-1')
        self.s3_bucket = self.s3.create_bucket(Bucket='some-bucket')

    def test_sth(self):
        self.assertTrue(True, "should arrive here")

requirements.txt:

boto3 == 1.9.115
botocore == 1.12.115
moto == 1.3.7

To run the test pls execute: python3 -m unittest discover -s . -p "*_test.py".

The result as I see is: botocore.errorfactory.BucketAlreadyExists: An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.

But if you don’t have AWS auth in the environment the error will be: botocore.exceptions.NoCredentialsError: Unable to locate credentials

The code above worked perfectly when using moto 1.3.7, boto3 1.7.84 and botocore 1.10.84 (the same was stated in this issue: #1815), for anything newer of boto causes moto to fail to mock properly. I’ve just spotted #1793 - very similar code sample and exactly the same problems.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 22 (2 by maintainers)

Most upvoted comments

@kgutwin - that’s strange… on another machine (also OSX) version of moto just right from git does not fix the problem. I assume that some other dependencies may interfere?

On dockerized version of this test it works, but when I modify the code to use SQS (self.request = self.sqs.create_queue(QueueName='queue')) I get:

botocore.errorfactory.QueueNameExists: An error occurred (QueueAlreadyExists) when calling the CreateQueue operation: The specified queue already exists.

So, even moto master has still problems with accessing AWS - this time for SQS and probably some other services, too.

pskowronek on Mar 20, 2019

A new release has been cut. Feel free to open an issue if there are still problems.

spulec on Jul 8, 2019

Unfortunately, moto declares that it is compatible with botocore 1.12.86 and above which prevents a tool like poetry to resolve dependencies… And no override option is available to fix this case: sdispater/poetry#697

aizelauna on Jun 20, 2019

localstack has also been rough lately. both projects need some fresh blood, sweat and tears.

adamdavis40208 on May 17, 2019

Is Moto dead? The last few months have really been rough, and mocking AWS stuff is becoming miserable. Credit where credit is due of course (AWS changing everything) - what do we do? Needing to use the deprecated decorators to prevent network access is salt in the wound.

chris-erickson on May 17, 2019

After reproducing your issue in an isolated Docker container (docker run -ti --rm -v $PWD:/app python:3 bash), I believe the problem you’re seeing was reported in #1924 and fixed in PR #1952. That PR has been merged to master, but there are some lingering concerns with its approach (#2058).

Could you try reinstalling moto from git?

pip install --upgrade git+https://github.com/spulec/moto.git

kgutwin on Mar 19, 2019