serverless: using disableLogs option with a single httpApi fails to provision IamS Role

When using the ‘disableLogs’ option with a function set up to use httpApi rather than the default ‘restApi’ (example configuration below), the resulting IAMS role is not created properly. The configuration is generated without any ‘Resources’.

A workaround is to define a second ‘fake’ function which does not have ‘disableLogs’ option. This allows the role to be correctly generated.

The bug was likely introduced here: https://github.com/serverless/serverless/pull/8561/files

As you can see by looking at the code, it sort of relies on at least one defined function being able to get past the ‘return’ here: https://github.com/serverless/serverless/pull/8561/files#diff-634c192175fa19e078211273e8cec92851e3a2fb117fdf447b6998fa1af1a5cfR104

So if at least 1 function is not set up to ‘log’ then the policy is attempted to be created with an invalid ‘Resource []’ node (it should never be empty)

e.g

"PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Action": [
                    "logs:CreateLogStream",
                    "logs:CreateLogGroup"
                  ],
                  "Resource": []
                },
                {
                  "Effect": "Allow",
                  "Action": [
                    "logs:PutLogEvents"
                  ],
                  "Resource": []
                }
              ]
            }
          }

I guess the next step would be to convert my provided serverless.yml into a failing test case?

serverless.yml

service: my-service
unresolvedVariablesNotificationMode: error
variablesResolutionMode: 20210219

provider:
  name: aws
  runtime: nodejs12.x
  region: ${opt:region}
  deploymentBucket:
    name: ssr-lambda-${opt:region}
  stage: ${opt:stage, self:custom.defaultStage}
  profile: ${self:custom.profiles.${self:provider.stage}}
  memorySize: 1024
  logRetentionInDays: 5
  lambdaHashingVersion: 20201221
  endpointType: REGIONAL
  apiGateway:
    shouldStartNameWithService: true

package:
  defaultStage: dev
  profiles:
    dev: devProfile

functions:
  ssr:
    handler: ./dist/index.handler
    disableLogs: true
    events:
      - httpApi: '*'

NODE_ENV=production webpack && sls deploy output
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service our-code.zip file to S3 (13.22 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
....
Serverless: Operation failed!
Serverless: View the full error output: snipped

 Serverless Error ----------------------------------------

  An error occurred: IamRoleLambdaExecution - Policy statement must contain resources. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: 29809a22-xxx-4988-a82d-xxxxxxx; Proxy: null).

Installed version

> NODE_ENV=production sls --version

Framework Core: 2.29.0 (local)
Plugin: 4.5.0
SDK: 4.2.0
Components: 3.7.3

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Reactions: 4
  • Comments: 15 (2 by maintainers)

Commits related to this issue

Most upvoted comments

So far we have decided to disableLogs on everything but one function that doesn’t emit many in order to stop paying for duplicate logging costs between CloudWatch and a third party we emit logs to directly from Lambda.

I’m really disappointed this feature has been broken for years and no hope in sight for a fix.

The issue that apparently solved this problem was closed almost one year ago but I’m still facing this error as well.

serverless.ts

import type { AWS } from '@serverless/typescript'

const serverlessConfiguration: AWS = {
  service: 'service-name',
  frameworkVersion: '3',
  plugins: [
    'serverless-esbuild',
    'serverless-offline',
    'serverless-offline-ssm',
    'serverless-plugin-datadog',
    'serverless-step-functions',
  ],
  // ...
  functions: {
    'function-name': {
      // ...
      disableLogs: true
    }
  }
  // ...
}

$ node --version v14.18.1

serverless: v3.17.0 serverless-esbuild: v1.27.1 serverless-offline: v8.7.0 serverless-offline-ssm: v6.2.0 serverless-plugin-datadog: v5.1.1 serverless-plugin-typescript: v2.1.2 serverless-step-functions: v3.7.0

Temporary solution

I’m not proud of it but I managed to make a workaround for this issue.

As the CloudFormation template is generated with wrong “Resource” values and it’s a JSON, I created a script that runs between sls package and sls deploy actions modifying the generated JSON CF Template removing those invalid resoures.

To do this:

  1. I installed the serverless-plugin-scripts (v1.0.2) plugin;
  2. Added this configuration to serverless.ts:
custom: {
    scripts: {
      hooks: {
        'package:finalize': 'node ./infra/scripts/fix_disableLogs_resource.js',
      },
    },
}
  1. Created a script to modify the generated .serverless/serverless-state.json file, used by sls deploy.
fix_disableLogs_resource.js
const fs = require('fs')
const path = require('path')

// =-=-=-=-=: Functions :=-=-=-=-=

function readJsonFile(fileName) {
  const cfStackTemplateFile = path.resolve(`.serverless/${fileName}`)

  return {
    path: cfStackTemplateFile,
    content: require(cfStackTemplateFile),
  }
}

function backupFile(file) {
  const bkpPath = `${file.path}.bkp`
  if (!fs.existsSync(bkpPath)) {
    fs.copyFileSync(file.path, bkpPath)
  }
}

function hasResource(statement) {
  return !!statement.Resource?.length || !!statement.$ref
}

function fixLambdaExecutionPolicies(template) {
  const iamLambdaExecutionResource = template.Resources.IamRoleLambdaExecution
  if (iamLambdaExecutionResource) {
    const iamRolePolicies = iamLambdaExecutionResource.Properties.Policies

    iamRolePolicies.forEach(policy => {
      const statements = policy.PolicyDocument.Statement
      const statementWithValidResource = statements.filter(hasResource)
      policy.PolicyDocument.Statement = statementWithValidResource
    })
  }
}

function writeFile(file) {
  fs.writeFileSync(file.path, JSON.stringify(file.content))
}

// =-=-=-=-=: Execution :=-=-=-=-=

console.log(
  "Updating compiledCloudFormationTemplate excluding IAM Role's policy empty resources...",
)

// Only for consistency's sake
const stackFile = readJsonFile('cloudformation-template-update-stack.json')
backupFile(stackFile)
fixLambdaExecutionPolicies(stackFile.content)
writeFile(stackFile)

// This changes the real deployed package
const stateFile = readJsonFile('serverless-state.json')
backupFile(stateFile)
fixLambdaExecutionPolicies(
  stateFile.content.service.provider.compiledCloudFormationTemplate,
)
writeFile(stateFile)

console.log('compiledCloudFormationTemplate template updated.')

This is not perfect, but works and solved my problem.

Hi there, any news on this? We’re also getting this error while using a simple handler for SQS events:

...
SimpleConsumer:
  handler: foo.bar.SimpleConsumer
  disableLogs: true
  timeout: 45
  events:
    - sqs:
        arn: !GetAtt SimpleQueue.Arn
  vpc:
    ${self:custom.vpcSettings}
...
> sls deploy -s dev
✖ Stack simple-be-dev failed to deploy (37s)
Environment: darwin, node 18.3.0, framework 3.18.2 (local) 3.8.0v (global), plugin 6.2.2, SDK 4.3.2
Credentials: Local, environment variables
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
UPDATE_FAILED: IamRoleLambdaExecution (AWS::IAM::Role)
Policy statement must contain resources. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: d569bf61-7aff-4491-b16b-6a23303374f4; Proxy: null)

Hi, Can someone guide me on disableLogs issue. I’m still facing it.

Hello @calumbrodie, thanks for reporting. I’ve managed to reproduce your issue locally. I believe in such situations we should not add the policy with logs:CreateLogStream, logs:CreateLogGroup and logs:PutLogEvents actions at all, it should ensure that the role can be created without issues.

We’d be happy to accept a PR that fixes that problem 👍

This is a really interesting issue, that I’ve been seeing since at least one year (serverless 3.21.0). Today I decided to spend a little more time on it. My case is the following: deploy a function to AWS with disableLogs: true fails. I’m not willing to go the custom script route, and looking at the closed PR discussion, I don’t think I can fix this myself. Here’s what I tried:

  • Removing the whole cloud formation stack and redeploying seems to work
    • I assume the same for commenting the functions, deploying, then uncommenting them and deploying
    • Deploying at least one function with disableLogs: false works
  • Renaming the functions doesn’t work
  • Adding an overriding iamRoleStatements rule with effect Allow or Deny doesn’t work
  • Adding “custom” names (instead of the “canonical” name) with the name attribute doesn’t work
  • Updating to the latest sls version (3.33.0) doesn’t work
  • Adding a dummy function with logs enabled, and then removing it doesn’t work.

So for now, I will have to always keep one of the functions with logs, unfortunately.

ServerlessError: An error occurred: IamRoleLambdaExecution - Policy statement must contain resources. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: cacab1ed-4684-48ad-b150-db73fd031d37; Proxy: null).
593 | at /codebuild/output/src687836509/src/service/node_modules/serverless/lib/plugins/aws/lib/monitor-stack.js:149:23
594 | at processTicksAndRejections (internal/process/task_queues.js:95:5)
595 | at AwsDeploy.createFallback (/codebuild/output/src687836509/src/service/node_modules/serverless/lib/plugins/aws/lib/update-stack.js:62:5) {
596 | code: 'AWS_CLOUD_FORMATION_CREATE_STACK_INTERNAL_I_A_M_ROLE_CREATE_FAILED',
597 | decoratedMessage: 'CREATE_FAILED: IamRoleLambdaExecution (AWS::IAM::Role)\n' +
598 | 'Policy statement must contain resources. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: cacab1ed-4684-48ad-b150-db73fd031d37; Proxy: null)\n' +
599 | '\n' +
600

Any update on this issue ?