boto3: generate_presigned_url no longer works, gives: SignatureDoesNotMatch

@imperio59 's answer helped, but I also needed to set signature_version='s3v4':

s3 = boto3.client('s3', aws_access_key_id={YOUR ACCESS KEY}, aws_secret_access_key={YOUR SECRET ACCESS},
                  config=Config(s3={'addressing_style': 'path'}, signature_version='s3v4'))

boto3 1.9.28 (1.9.34 is latest available) botocore 1.12.28 (1.12.34 is latest available) python 3.6.4 (MacOS)

Thanks @Trogious ! We had the same issue SignatureDoesNotMatch uploading an image from our React Native APP.

I fixed it passing the signature_version='s3v4' and the region_name

s3 = session.client('s3',
                    config=boto3.session.Config(signature_version='s3v4'),
                    region_name='eu-central-1'
)

After several hours of hitting this same issue, changing the addressing_style to ‘path’ made everything work for me, including custom headers…

See docs here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3.html#changing-the-addressing-style

I needed BOTH s3v4 and addressing_style:path in my Config() for it to work with boto3.

from botocore.config import Config
my_config = Config(
    region_name = 'us-east-2',
    signature_version = 's3v4',
    retries = {
        'max_attempts': 10,
        #'mode': 'standard'
    },
    s3={'addressing_style': 'path'},
)

… and then the functional part …

client = boto3.client('s3', config=my_config,
    aws_access_key_id = AWS_ACCESS_KEY_ID,
    aws_secret_access_key = AWS_SECRET_ACCESS_KEY
)

bucket_name = <bucket name with no leading or trailing slashes>
key_name = <look up your `key` in the console for this S3 object, not the URL>

response = client.generate_presigned_url(
    'get_object',
    Params={'Bucket': bucket_name, 'Key': key_name},
    ExpiresIn=604800,
    HttpMethod=None
)

print(response)

Not sure how to make the URL last 7 days as requested. That has to do with the IAM I used, but working on it.

Without addressing_style: path I get this log:

Event before-parameter-build.s3.GetObject: calling handler <function sse_md5 at 0x7f5cc39fccb0>
Event before-parameter-build.s3.GetObject: calling handler <function validate_bucket_name at 0x7f5cc39fcc20>
Event before-parameter-build.s3.GetObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f5cacb5bc90>>
Event before-parameter-build.s3.GetObject: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x7f5caceac4d0>>
Event before-parameter-build.s3.GetObject: calling handler <function generate_idempotent_uuid at 0x7f5cc39fca70>
Event choose-signer.s3.GetObject: calling handler <function set_operation_specific_signer at 0x7f5cc39fc950>
Event before-sign.s3.GetObject: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x7f5cacb61550>>
Defaulting to S3 virtual host style addressing with path style addressing fallback.
Checking for DNS compatible bucket for: https://s3.eu-central-1.amazonaws.com/bucket/file.txt
URI updated to: https://bucket.s3.amazonaws.com/file.txt
Calculating signature using v4 auth.
CanonicalRequest:
GET
/file.txt
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=MYKEY%2F20201119%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20201119T141500Z&X-Amz-Expires=7200&X-Amz-SignedHeaders=host
host:bucket.s3.amazonaws.com

host

Which generates this url:

https://bucket.s3.amazonaws.com/file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=MYKEY%2F20201119%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20201119T141500Z&X-Amz-Expires=7200&X-Amz-SignedHeaders=host&X-Amz-Signature=signature

Which when opened redirects to this url:

https://bucket.s3.eu-central-1.amazonaws.com/file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=MYKEY%2F20201119%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20201119T141500Z&X-Amz-Expires=7200&X-Amz-SignedHeaders=host&X-Amz-Signature=signature

Where CanonicalRequest changes the host too:

GET 
/file.txt 
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=MYKEY%2F20201119%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20201119T141500Z&X-Amz-Expires=7200&X-Amz-SignedHeaders=host
host:bucket.s3.eu-central-1.amazonaws.com

host
UNSIGNED-PAYLOAD

For some reason the code uses host:bucket.s3.amazonaws.com while S3 expects host:bucket.s3.eu-central-1.amazonaws.com.

When I initialize the client with region specified, the local canonical host is still the same: host:bucket.s3.amazonaws.com

Edit: got it working with config=Config(s3={"addressing_style": "virtual"})

@lu1s There’s nothing inherently wrong with either of those URLs you posted previously, just that they’re using two different signature versions.

It is possible that switching from virtual to path addressing style could fix pre-signed URLs signed using SigV4. For SigV4 the host is signed as part of the signature, which can cause problems for newly created buckets when virtual addressing style is being used. S3 will redirect to a different host for buckets that DNS haven’t propagated for, which leads signature mismatch errors as the host is no longer correct. My hunch is that this is the case most people are running into.

It’s also worth nothing that some regions only support SigV4 and you’ll need to ensure that you’re using it for those regions.

Using the latest version of boto3 I was able to successfully generate and use a presigned URL created with all permutations of SigV2/SigV4 and path/virtual addressing style for a bucket that existed for quite some time.

Let me know if that clears things up.