moto: moto 1.3.8 breaks real boto3 access

Just installed the latest 1.3.8 release and it appears to have broken my tests that make actual non-mocked STS calls when the test file includes moto. I’ve narrowed down the issue and created a minimal example to show the problem.

Test code in file moto38.py:

import json
import boto3
from moto import mock_sts


policy = {
    'Version': '2012-10-17',
    'Statement': [
        {
            'Effect': 'Allow',
            'Action': '*',
            'Resource': '*'
        }
    ]
}

sts = boto3.client('sts')
response = sts.assume_role(
    RoleArn='arn:aws:iam::12345678:role/xxx',
    RoleSessionName='yyy',
    Policy=json.dumps(policy),
    DurationSeconds=900
)

Now run the test and note the error output with exception InvalidClientTokenId:

Traceback (most recent call last):
  File "/Users/dmulter/Desktop/moto38.py", line 22, in <module>
    DurationSeconds=900
  File "/Users/dmulter/Documents/projects/xxx/.virtualenv/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/dmulter/Documents/projects/xxx/.virtualenv/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidClientTokenId) when calling the AssumeRole operation: The security token included in the request is invalid.

If I fall back to moto==1.3.7 the test gets past this error. Note that I didn’t bother making the policy and role work properly, but in my tests the real values work as expected when using the previous moto version.

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 2
Comments: 18 (2 by maintainers)

Commits related to this issue

Move env variable mocking and undo when stopping. CC #2058, #2172. — committed to getmoto/moto by spulec 5 years ago
Move env variable mocking and undo when stopping. CC #2058, #2172. — committed to efiop/moto by spulec 5 years ago
Merge pull request #2285 from spulec/move-env-mocking Move env variable mocking and undo when stopping. CC #2058, #2172. — committed to getmoto/moto by spulec 5 years ago

Most upvoted comments

Sorry for my admittedly terse response.

I think it is a normal expectation that packages do stable releases at some periodic frequency.

I’ve been having discussions with various people and trying to really understand the value of this for Moto. Unlike a normal package, we are always adding new endpoints/resources. When we cut releases, there isn’t anything additional that is done to ensure it is “stable”. The releases aren’t any different than the prereleases.

I’ll work to cut a release in the next couple days, but I do think our current process is broken. I’m tempted to either have every commit make a real release or do something like pytz and have monthly releases (we would automate them). My fear with both of these is giving a false sense of security to people.

Thoughts?

spulec on Oct 31, 2019

@spulec when is this fix going to be released to PyPi? because pulling from moto==“*” does not pull the #2285 PR. We had no choice than to lock down our moto package to the master branch which is definitely not healthy

ch3ck on Jul 23, 2019

Great. I’ve merged it: https://github.com/spulec/moto/pull/2285

spulec on Jul 20, 2019

It also breaks real boto (2) access. I believe the root cause is this commit - https://github.com/spulec/moto/commit/cf5bd7665cff971b8f0e470e96773bcbe1e5ad36, setting 2 env variables (AWS_ACCESS_KEY_ID , AWS_SECRET_ACCESS_KEY) in moto/core/models.py. They can impact boto connection initialization on every flow running after the models.py is imported.

This works fine: >>> import boto >>> conn = boto.connect_s3() >>> conn.access_key

'XXXXXXX'

>>> import moto >>> conn.access_key

'XXXXXXX'

>>> conn.get_bucket('mybucket')

<Bucket: mybucket>

While this breaks: >>> import boto >>> import moto >>> conn = boto.connect_s3() >>> conn.access_key 'foobar_key'

>>> conn.get_bucket('mybucket') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "...lib/python3.6/site-packages/boto/s3/connection.py", line 509, in get_bucket return self.head_bucket(bucket_name, headers=headers) File "...lib/python3.6/site-packages/boto/s3/connection.py", line 542, in head_bucket raise err boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden

tomerben on Jun 23, 2019

I would suggest that they are all official releases then. I agree that a monthly “official” release would present a false sense of stability of the release. The boto3 and botocore packages have the same (and even larger) frequency of updates, and they are all official releases. I think that’s a better parallel with moto releases. I would rather detect issues with a specific release and pin the version if I run into any issues, which I have to do in rare cases with boto3. This is a common pattern.

dmulter on Jan 2, 2020

pip install -U --pre moto ?

spulec on Oct 31, 2019

Pre-release packages are fine, but then I have to pin that specific version. Is there any plan for when the next official release is expected? I would like to keep just pip install -U moto.

dmulter on Oct 24, 2019

Can someone please try the PR here? https://github.com/spulec/moto/pull/2285

spulec on Jul 8, 2019