amplify-js: Storage.put, Storage.get with protected images give 403 error out of the box on localhost

I am getting a 403 when using Storage.put as well, which started when I upgraded to latest versions of amplify-js…

If a user is part of a group , it uses groupRole rather than AuthRole.

Is this somewhere in the documentation? (If not, could it be added?)

I also interpreted the CLI prompts to imply that setting up “Authorized users” would include all authorized users, regardless of Group - especially when I did not set up Group authentication. It is not clear from the prompts that a group role will replace an auth role.

I can reproduce this even if I add hosting and deploy the react app … I am using @auth(rules: [{ allow: groups, groups: ["Admin"], operations: [create update delete] }]) and I get a 403 on upload to S3. What’s even more concerning is that if the user that I am logged in with is not part of the Admin group then the mutation fails but the upload is successful … yikes.

Even with the most minimalistic auth resource added to the model i.e. { allow: owner }, I get a 403 when uploading.

For what it is worth, I am facing this issue too.

I get the errors for public and private files too.

@danrivett Ah, I see your confusion here. I’ll dive into some details out here for explaining this behavior. TLDR - Groups do not inherit permissions from the auth permissions/role. Groups have their own set of permissions which are completely independent of the auth and guest users permissions. Explanation: When we add a Cognito user pool group using the CLI - each group gets an IAM role attached to it with the least set of permissions (empty to begin with) which is independent of the role assumed by the auth and guests users. Unless you explicitly add permissions for that group for a given category (using the amplify update category flow) the permissions don’t get get propagated to the group permissions. One of the things we could expose in the CLI is the ability for customers to inherit the permissions from the auth permissions/role and add it to the group role - as I’ve proposed out here as well - https://github.com/aws-amplify/amplify-js/issues/5729#issuecomment-635728670 The downside to this is that now the groups will always have all the permissions tied to the auth user - which some customers might not want - hence I would consider this as an advanced option and not the default.

Regarding the options out here: ? Restrict access by? (Use arrow keys) ❯ Auth/Guest Users Individual Groups Both

If you choose Auth/Guest users you’ll be presented with options to select from the CRUD permissions for the Auth and Guest users only - the Group roles would still have no permissions. If you select “Individual Groups” - you’ll be presented with options to select from the CRUD permissions for each individual group. So you can provide say “Read” permissions to one of the groups and just “Create” permissions to another group. When you select “Both” - the CLI walks you through configuring the permissions for Auth, Guest Users as well as Individual Groups (so it’s basically a combination of the first and second option).

Regarding the primary pain point which you’ve mentioned out here - if a user gets added to a group outside of Amplify and if you want all the permissions to be inherited from the auth user- then the group IAM role has to also be configured to have permissions to perform all the actions of auth users or if you want to add any custom permissions for the group - you can use the custom policy functionality which we expose. You can read more about it here - https://docs.amplify.aws/cli/auth/groups#group-access-controls

I also have an explanation to your comment out here - https://github.com/aws-amplify/amplify-js/issues/5729#issuecomment-628014869

However even if it is designed to not work like that, if I don't configure any Storage permissions for a particular unrelated Cognito Group, then it should ignore it and just use the other configured Storage permissions, which in my case would be regular owner permissions since the user was neither in the 'Adminstrators' or 'Viewers' group

The reason for this again is because Storage category uses IAM roles for its permissions and if the corresponding group role doesn’t have permissions configured to inherit permissions from the auth role - then the users in the group wouldn’t be able to perform the operations which the auth users are able to perform. Exposing the advanced option which I’ve proposed above - would solve for this issue as well.

Also regarding comparison with @auth, let’s take your example in the comment https://github.com/aws-amplify/amplify-js/issues/5729#issuecomment-628014869 and see how you can achieve this with storage category permissions

@Auth annotation. For example my model is annotated as such:

type Worker @model
  @auth(rules: [
    { allow: owner },
    { allow: groups, groups: ["Administrators"] },
    { allow: groups, groups: ["Viewers"], operations: [read] }

This provides:

• The owner with CRUD access but only to its own Worker

If the user uploads a file using the private level of protection from the client, the the user has options to perform CRUD given CRUD permissions are configured for the auth users in the CLI flow.

• Administrators with CRUD access for all

It’s not currently possible for a auth users or any groups to perform CRUD on all the files i.e all files stored in protected/, private/ and public/. Only the files stored using public protection can be read/updated/deleted by auth or groups depending on the CRUD permissions configured.

• Viewers with only read access to all (and CRUD access to its own Worker of course) In this case the viewers group can be configured to have only “Read” operations and you can use the protected/ level to upload the files from the client

As you see above - the permissions for the files stored using the Storage category rely both on the client-level protections(public/private/protected) as well as the backend configurations (CRUD operations) set by the CLI unlike the API level protections which is strictly backend configurations - making it a bit difficult to draw a 1:1 corollary. At the same time, I would say our team is discussing this more in depth to see if we can make this flow simpler and more intuitive and similar to the api flows.

Hi @danrivett

Thank you for your patience and apologies for the delay. We are on top of this and will circle back to you with updates.

If a user is part of a group , it uses groupRole rather than AuthRole.

Is this somewhere in the documentation? (If not, could it be added?) I also interpreted the CLI prompts to imply that setting up “Authorized users” would include all authorized users, regardless of Group - especially when I did not set up Group authentication. It is not clear from the prompts that a group role will replace an auth role.

@taylornewton is there a way to retrospectively change this in your amplify S3 config?

@matt-d-webb If I remember correctly, I manually updated the permissions in S3 to allow the Admin group role to have permissions to the bucket. I did not do anything with the Amplify config.

Hi @taylornewton

Thanks for reporting this. We have taken up this task in our queue to improve the documentation so that this confusion can be avoided.

Let me know if anyone is still stuck on this issue.

@akshbhu Secondly, after fixing the first bug we would actually like to configure Storage with the ‘Both’ option to allow administrators CRUD access to all the storage, viewers read-only access to all storage, and any other user not part of those groups only access to their own storage using the standard ‘private’, ‘protected’, and ‘public’ access levels the Storage component provides.

Maybe that’s not how the Storage component works, and is a misunderstanding on my part, but those expectations are based on how group permissions work with API component and the @Auth annotation. For example my model is annotated as such:

type Worker @model
  @auth(rules: [
    { allow: owner },
    { allow: groups, groups: ["Administrators"] },
    { allow: groups, groups: ["Viewers"], operations: [read] }

This provides:

• The owner with CRUD access but only to its own Worker
• Administrators with CRUD access for all
• Viewers with only read access to all (and CRUD access to its own Worker of course)

So I expect the Storage component to work in the same manner.

However even if it is designed to not work like that, if I don’t configure any Storage permissions for a particular unrelated Cognito Group, then it should ignore it and just use the other configured Storage permissions, which in my case would be regular owner permissions since the user was neither in the ‘Adminstrators’ or ‘Viewers’ group.

This would mean that the user could do a private, protected or public PUT using its owner credentials. However that is not currently the case, it gets a 403 because it’s trying to use a group membership that has not been configured in the Storage component and is completely unrelated to providing Storage permissions.

@akshbhu I think the first bug I’m seeing here is that cognito group membership should not even be considered by the Storage component when I do not configure the Storage component on the CLI to use Group authentication but instead just allow ‘authorized’ (non-guest) users:

? Restrict access by? Auth/Guest Users
? Who should have access: Auth users only

I did not choose ‘Both’ or Group access, so it should not care that the user happens to be part of a Cognito Group. So that’s a bug from my understanding. Users may be part of groups for reasons outside of Amplify or certainly Storage permissions, and that should not break them here.

I would definitely appreciate if you could recreate this yourself as it should be very simple to recreate, and I think it will help in understanding where this first bug lies.

@Amplifiyer I have confirmed my issue is related to Cognito User Groups since I tried the same PUT request with a user that has no Group memberships and it succeeded. So the question is why is the fact a user is a member of a Group getting in the way here?

Here’s my CloudTrail redacted log with no error reported this time:

{
    "eventVersion": "1.07",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "***:CognitoIdentityCredentials",
        "arn": "arn:aws:sts::***:assumed-role/amplify-personnel-dan-***-authRole/CognitoIdentityCredentials",
        "accountId": "***",
        "accessKeyId": "***",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "***",
                "arn": "arn:aws:iam::***:role/amplify-personnel-dan-***-authRole",
                "accountId": "***",
                "userName": "amplify-personnel-dan-***-authRole"
            },
            "webIdFederationData": {
                "federatedProvider": "cognito-identity.amazonaws.com",
                "attributes": {
                    "cognito-identity.amazonaws.com:amr": "[\"authenticated\",\"cognito-idp.us-west-2.amazonaws.com/us-west-2_***\",\"cognito-idp.us-west-2.amazonaws.com/us-west-2_***:CognitoSignIn:***\"]",
                    "cognito-identity.amazonaws.com:aud": "us-west-2:***",
                    "cognito-identity.amazonaws.com:sub": "us-west-2:***"
                }
            },
            "attributes": {
                "creationDate": "2020-05-11T22:28:13Z",
                "mfaAuthenticated": "false"
            }
        }
    },
    "eventTime": "2020-05-11T22:29:36Z",
    "eventSource": "s3.amazonaws.com",
    "eventName": "PutObject",
    "awsRegion": "us-west-2",
    "sourceIPAddress": "***",
    "userAgent": "[Expo/2.15.4.10229 CFNetwork/1125.2 Darwin/19.4.0]",
    "requestParameters": {
        "bucketName": "personnel***-dan",
        "Host": "personnel***-dan.s3.us-west-2.amazonaws.com",
        "key": "public/test-public.txt",
        "x-id": "PutObject"
    },
    "responseElements": null,
    "additionalEventData": {
        "SignatureVersion": "SigV4",
        "CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
        "bytesTransferredIn": 5.0,
        "AuthenticationMethod": "AuthHeader",
        "x-amz-id-2": "***",
        "bytesTransferredOut": 0.0
    },
    "requestID": "D40C14C33283A7EB",
    "eventID": "2bb90f3c-6109-48e6-8c8d-c2b44f1e9266",
    "readOnly": false,
    "resources": [
        {
            "type": "AWS::S3::Object",
            "ARN": "arn:aws:s3:::personnel***-dan/public/test-public.txt"
        },
        {
            "accountId": "***",
            "type": "AWS::S3::Bucket",
            "ARN": "arn:aws:s3:::personnel***-dan"
        }
    ],
    "eventType": "AwsApiCall",
    "managementEvent": false,
    "recipientAccountId": "***",
    "eventCategory": "Data"
}

Notice that the userName now ends in authRole?

Regarding recreating, it’s very hard to create a simple recreate since I’m recreating using a React Native application connected to an Amplify provisioned backend.

I would suggest you may be able to recreate it yourself by:

• Adding the user you’re authenticating with to a Cognito Group first
• Then reauthenticating again after a few minutes has passed
  • (To allow the Group membership in Cognito to take effect since I don’t know if this happens instantaneously or in an eventually-consistent manner)
• Then reattempting your PUT.
  • Hopefully you’ll get a 403 this time.

@akshbhu thinking about this more, sending sensitive information via email isn’t much more secure to be honest - there’s a reason why banks never do this as totally vulnerable in transit. The amplify directory contains authentication data so I’m a bit uncomfortable sending it via email even if it is non-prod.

For prod it would be a non-starter so there must be a better mechanism to send you the amplify directory? Does AWS have an upload page that I can upload the zip via rather than sending it via email?

Hi @danrivett

The IAM policy has protected/${cognito-identity.amazonaws.com:sub}/* but that value is not the sub of the current user id.

Yes you are correct, this is the Cognito IdentityId which is different from the User sub which you get from user Pool access tokens. Technically, you wouldn’t need Cognito user-pools to authenticate against S3.

put test.txt to protected/us-west-2:2723a679-65cb-440d-a3bb-155b7c209241/test.txt

So this PUT request should work as it is using Identity ID.

I think your issue is a bit different from the one above , you can open an issue here .

Regarding submitting issues you can mail you amplify folder zip to amplify-cli@amazon.com. Please DO NOT post sensitive information on public page. We can reproduce this issue using your amplify folder.

As mentioned, you can store it against an object at upload time to dynamodb.

type S3Object {
  key: String
  level: String
  identityId: String
}

Then

Storage.get(key, {level, identityId})

I agree it’s torture but it’s the way they’ve built it.

I found the problem: the Storage.get() inserts “/public/” in between S3 bucket endpoint and the S3 key in the returned pre-signed url. For an image file when using this pre-signed url as the image src you will get the 403 forbidden error (it should return 404 file not found error, instead). I tried to work around the problem by removing the “/public/” from the presigned url. However, I got an error “SignatureDoesNotMatch” return because it is a signed url I cannot change it. If I create a “public” folder in my S3 bucket and move my image files into the “public” folder then it works. Is there anyway to configure the Store sub-module not insert the “/public/” in the returned url?

For those who wishes to use an Admin user group which has access to all other users’ uploads, I found out a temporary solution by manually modifying storage config files.

We’d modify in total 3 files to grant proper permissions to the Admin user group:

• backend-config.json, under ./amplify/backend
• s3-cloudformation-template.json, under ./amplify/backend/storage/YourStorageResourceName
• storage-params.json, under ./amplify/backend/storage/YourStorageResourceName

Replace all YourAuthResourceNameHere in the following templates with your authentication id, typically this is the folder name under ./amplify/backend/auth/. And all YourStorageResourceNameHere with your storage resource name, typically this is the folder name under ./amplify/backend/storage.

In backend-config.json, add the following dependsOn section into your storage resource:

    "storage": {
        "YourStorageResourceNameHere": {
            "service": "S3",
            "providerPlugin": "awscloudformation",
            "dependsOn": [
                {
                    "category": "auth",
                    "resourceName": "YourAuthResourceNameHere",
                    "attributes": [
                        "UserPoolId"
                    ]
                },
                {
                    "category": "auth",
                    "resourceName": "userPoolGroups",
                    "attributes": [
                        "AdminGroupRole"
                    ]
                }
            ]
        }
    },

In s3-cloudformation-template.json, append the following to Resources:

        "AdminGroupPolicy": {
            "Type": "AWS::IAM::Policy",
            "Properties": {
                "PolicyName": "Admin-group-s3-policy",
                "Roles": [
                    {
                        "Fn::Join": [
                            "",
                            [
                                {
                                    "Ref": "authYourAuthResourceNameHereUserPoolId"
                                },
                                "-AdminGroupRole"
                            ]
                        ]
                    }
                ],
                "PolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Action": [
                                "s3:PutObject",
                                "s3:GetObject",
                                "s3:ListBucket",
                                "s3:DeleteObject"
                            ],
                            "Resource": [
                                {
                                    "Fn::Join": [
                                        "",
                                        [
                                            "arn:aws:s3:::",
                                            {
                                                "Ref": "S3Bucket"
                                            },
                                            "/*"
                                        ]
                                    ]
                                }
                            ]
                        },
                        {
                            "Effect": "Allow",
                            "Action": "s3:ListBucket",
                            "Resource": [
                                {
                                    "Fn::Join": [
                                        "",
                                        [
                                            "arn:aws:s3:::",
                                            {
                                                "Ref": "S3Bucket"
                                            }
                                        ]
                                    ]
                                }
                            ]
                        }
                    ]
                }
            }
        }

And finally, create a new file storage-params.json under ./amplify/backend/storage/YourStorageResourceName:

{
    "groupPermissionMap": {
        "Admin": [
            "create/update",
            "read",
            "delete"
        ]
    }
}

I’m still investigating, but I’m also finding the following isn’t working for me:

await Storage.put('test.txt', 'Protected Content', {
      level: 'protected',
      contentType: 'text/plain',
    });

So I don’t think this particular issue is related to blobs. I’m getting a 403 for that too.

What I’m finding odd, and perhaps it’s correct, but seems suspicious is the path it’s trying to PUT to:

[DEBUG] 16:28.624 AWSS3Provider - put test.txt to protected/us-west-2:2723a679-65cb-440d-a3bb-155b7c209241/test.txt

Does that path look correct? The IAM policy has protected/${cognito-identity.amazonaws.com:sub}/* but that value is not the sub of the current user id. Here’s more debug logging you can see the log of the current user which shows the sub as f0bc9559-a0e1-408f-9ad0-45f5b379f98d but that’s not what is being used in the S3 path.

[DEBUG] 15:54.81 AuthClass - Succeed to get the user session CognitoUserSession {
  "accessToken": CognitoAccessToken {
    "jwtToken": "***",
    "payload": Object {
      "auth_time": 1588979621,
      "client_id": "36dq980j6m4mdmldjkdfa706eh",
      "cognito:groups": Array [
        "Workers",
      ],
      "event_id": "cb73749a-74f9-4cc3-9b30-1f10f41349d3",
      "exp": 1588983221,
      "iat": 1588979621,
      "iss": "https://cognito-idp.us-west-2.amazonaws.com/us-west-2_qzYQxS5Ab",
      "jti": "125cd2c9-53bd-447a-9edd-e65f7ebbc716",
      "scope": "aws.cognito.signin.user.admin",
      "sub": "f0bc9559-a0e1-408f-9ad0-45f5b379f98d",
      "token_use": "access",
      "username": "f0bc9559-a0e1-408f-9ad0-45f5b379f98d",
    },
  },
  "clockDrift": 0,
  "idToken": ...,
  "refreshToken": ...,
}

DAN: Current credentials: {"accessKeyId":"***","secretAccessKey":"***","sessionToken":"***","expiration":"2020-05-09T00:15:53.000Z","identityId":"us-west-2:2723a679-65cb-440d-a3bb-155b7c209241","authenticated":true}

...

[DEBUG] 16:28.613 AWSS3Provider - set credentials for storage Object {
  "accessKeyId": "***",
  "authenticated": true,
  "identityId": "us-west-2:2723a679-65cb-440d-a3bb-155b7c209241",
  "secretAccessKey": "***",
  "sessionToken": "***",
}

...

[DEBUG] 16:28.624 AWSS3Provider - put test.txt to protected/us-west-2:2723a679-65cb-440d-a3bb-155b7c209241/test.txt

@akshbhu what do you need here to move this one forward? There is a “not reproducible” tag however plenty of people have reproduced it with instructions.

That’s the identityId of the user who uploaded the file. Also by design. When you retrieve it you need to pass in the identityId. I store this against my S3Object in dynamodb as it’s the only way to reliably retrieve it. No idea if that’s a security flaw or not.

I also randomly started getting 403 on Storage.put for images/blobs a few days ago running aws-amplify@3.0.5-unstable.4

I could still upload a text string using Storage.put and it would be going into the right bucket but failed for images.

Just downgraded to 2.1.1 and it is now working fine again.