serverless: Enabling XRay tracing on an existing stack fails

Bug Report

Description

When enabling AWS X-Ray tracing on an existing stack, the update fails with the following error:

An error occurred: HelloLambdaFunction - The provided execution role does not have permissions to call PutTraceSegments on XRAY (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: ********-****-****-****-************).

This error does not seem to be reproducible 100% of the time and appears to be a race to update the Lambda Execution role and the function itself.

These CloudFormation logs show the IamRoleUpdate commenced after the function configuration update: image

  1. What did you do?

    • Deploy a Serverless stack with tracing disable
    • Update provider.tracing.lambda to true
    • Re-deploy the stack
  2. What happened?

    • The error The provided execution role does not have permissions to call PutTraceSegments on XRAY is returned and the stack does not update
  3. What should’ve happened?

    • The stack updates, grants the necessary permissions to the execution role, and enables Lambda X-Ray tracing
  4. What’s the content of your serverless.yml file?

    • Initial stack creation:
    service:
      name: xray-permissions
    
    provider:
      name: aws
      runtime: nodejs10.x
    
    functions:
      hello:
        handler: handler.hello
        events:
          - http:
              method: get
              path: hello
    
    • Update:
    service:
      name: xray-permissions
    
    provider:
      name: aws
      runtime: nodejs10.x
      tracing:
        lambda: true # This has been enabled after creation
    
    functions:
      hello:
        handler: handler.hello
        events:
          - http:
              method: get
              path: hello
    
  5. What’s the output you get when you use the SLS_DEBUG=* environment variable (e.g. SLS_DEBUG=* serverless deploy)

    • I didn’t manage to capture this debug information but will update the information when I can reproduce the issue reliably.

Similar or dependent issues:

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 1
  • Comments: 19 (9 by maintainers)

Commits related to this issue

Most upvoted comments

@jamesmbourne thanks for report. Technically this can happen when you provide a custom execution role (via provider.role or via role directly in a function configuration).

Can you confirm it’s not the case?

As, if default role is used, exactly those permissions are set: https://github.com/serverless/serverless/blob/55c259f994b0062d60c64407f0bec6f17ef44d57/lib/plugins/aws/package/compile/functions/index.js#L319-L332