serverless: An error occurred: ServerlessDeploymentBucket - API: s3:SetBucketEncryption Access Denied.

This is a Bug Report

Description

What went wrong? All our long running serverless stacks have been erroring out on deploys the last few days with:

An error occurred: ServerlessDeploymentBucket - API: s3:SetBucketEncryption Access Denied.

What did you expect should have happened? Deploy as per usual.
What was the config you used? Command:

serverless deploy -v --stage dev

What stacktrace or error message from your provider did you see?

Serverless: Zip service: /home/travis/build/stellr-net/lambda-xyz/.webpack/service [179 ms]
Serverless: Packaging service...
Serverless: Remove /home/travis/build/stellr-net/lambda-xyz/.webpack
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service lambda-xyz.zip file to S3 (1.16 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - lambda-xyz
CloudFormation - UPDATE_IN_PROGRESS - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - UPDATE_FAILED - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - UPDATE_ROLLBACK_IN_PROGRESS - AWS::CloudFormation::Stack - lambda-xyz-dev
CloudFormation - UPDATE_COMPLETE - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - lambda-xyz-dev
CloudFormation - UPDATE_ROLLBACK_COMPLETE - AWS::CloudFormation::Stack - lambda-xyz
Serverless: Operation failed!
Serverless: View the full error output: https://eu-central-1.console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stack/detail?stackId=arn%3Aaws%3Acloudformation%3Aeu-central-1%3A940181270536%3Astack%2Flambda-xyz-dev%2Ffcd88450-2c3f-11e8-a8de-500c44f62262
 
  Serverless Error ---------------------------------------
 
  An error occurred: ServerlessDeploymentBucket - API: s3:SetBucketEncryption Access Denied.
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information -----------------------------
     OS:                     linux
     Node Version:           8.15.1
     Serverless Version:     1.38.0

Similar or dependent issues: None

Additional Data

Code and IAM policies did not change. We managed to resolve this by removing the stack completely and re-deploying.

Serverless Framework Version you’re using: 1.38.0
Operating System: Linux
Stack Trace: As per above
Provider Error messages: As per above

About this issue

Original URL
State: open
Created 5 years ago
Comments: 20 (2 by maintainers)

Commits related to this issue

BUG: Admin group needs new s3 encryption permissions. (#38) https://github.com/serverless/serverless/pull/5800 introduced default s3 deploy bucket encryption, creating upstream bugs like https://gith... — committed to FormidableLabs/terraform-aws-serverless by ryan-roemer 5 years ago

Most upvoted comments

I use permissions bellow:

- s3:GetEncryptionConfiguration
- s3:PutEncryptionConfiguration

https://forums.aws.amazon.com/thread.jspa?messageID=827867

+24

jincod on Mar 18, 2019

@ivanbreet @manfioLP i copy & paste the https://github.com/serverless/serverless/pull/5800#issuecomment-629014695

====

Finally I got the root cause of this issue in my environment.

Our company puts the organization SCP (Service control policies) to all sub OU accounts. Because of that, we have no permssion of s3:PutEncryptionConfiguration even I am in admin group/roles.

It will be hard to get the approval to release this block in short term (You know that’s corp level’s security review, it takes time), second it is a best practices of AWS Landing zone default SCP policy, in fact, security team would not approve this permission if I asked for.

We need rethink this PR and make AES256 encryption and its extra permissions optional. That’s the main problem we need fix here.

===

so because this feature to be set as hard coding , it can’t be optional , i have no chance to set yes/no on it, i can’t implement any serverless framework projects in my company, because the infrastructure team uses aws loading zone and set the SCP (service control policy)

Sadly, i have to switch to AWS SAM now

please seriously fix this issue.

+11

ozbillwang on Jun 11, 2020

Is there any news about this problem? I think I’m having the same error trying to push backend changes to AWS in a serverless application via Amplify CLI.

I have Amplify CLI v. 4.50.2 installed

simone-marcon on May 12, 2021

Hi @ozbillwang, the issue we experienced was only on our existing lambda stacks. Adding s3:PutBucketAcl, s3:GetEncryptionConfiguration, s3:PutEncryptionConfiguration policies to our CI/CD users solved it for us. We also have not seen the issue since.

All our stacks created after the event also seems to be okay. If all fails, maybe try deploying a new stack or change the deployment bucket and see if the issue persists.

ivanbreet on May 15, 2020

looks all serverless examples are broken with this error.

ozbillwang on May 5, 2020

2 years without a fix for this? My application is also now unbuildable due to this defect.

9037568 on Jun 2, 2021

I got a similar error when upgrading from Amplify 4.44.2 to 5.0.2 and resolved it by adding these permissions:

s3:GetEncryptionConfiguration
s3:PutEncryptionConfiguration

paolo573 on Jul 1, 2021

@ivanbreet @jincod

Could you give details on how to do the change? I have given the administrator permission to my account, I still get this error when deploy the serverless example

https://github.com/serverless/examples/tree/master/aws-node-rest-api-with-dynamodb

I used to deploy before and it is fine.

ozbillwang on May 5, 2020