serverless: Deploy failed to subscribe function to existing SNS Topic

This is a Bug Report

Description

For bug reports:

functions:
  dispatcher:
    handler: dispatcher.dispatch
    events:
      - sns:
          arn: arn:xxx
  • What stacktrace or error message from your provider did you see? N/A deploy completed with success:
$ serverless deploy --verbose
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
CloudFormation - CREATE_IN_PROGRESS - AWS::CloudFormation::Stack - geocoder-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - CREATE_IN_PROGRESS - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - CREATE_COMPLETE - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - CREATE_COMPLETE - AWS::CloudFormation::Stack - geocoder-dev
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (10.08 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - geocoder-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - GeocodeLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - GeocodeLogGroup
CloudFormation - CREATE_COMPLETE - AWS::Logs::LogGroup - GeocodeLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - GeocodeLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - GeocodeLambdaFunction
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Function - GeocodeLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - GeocodeLambdaVersionRmRziP4pdOkGzNTTPDps1BAxX6QUsEImdKe9Yyn7qk
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - GeocodeLambdaVersionRmRziP4pdOkGzNTTPDps1BAxX6QUsEImdKe9Yyn7qk
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Version - GeocodeLambdaVersionRmRziP4pdOkGzNTTPDps1BAxX6QUsEImdKe9Yyn7qk
CloudFormation - UPDATE_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - geocoder-dev
CloudFormation - UPDATE_COMPLETE - AWS::CloudFormation::Stack - geocoder-dev
Serverless: Stack update finished...
Service Information
service: geocoder
stage: dev
region: eu-west-2
stack: geocoder-dev
api keys:
  None
endpoints:
  None
functions:
  geocode: geocoder-dev-geocode

Stack Outputs
ServerlessDeploymentBucketName: geocoder-dev-serverlessdeploymentbucket-a9oksuw16rmp
GeocodeLambdaFunctionQualifiedArn: arn:aws:lambda:eu-west-2:xxxxxx:function:geocoder-dev-geocode:1

Additional Data

  • Serverless Framework Version you’re using: 1.25.0
  • Operating System: Windows
  • Node: 6.10.3

Please note that the first example in the docs DOES work:

functions:
  dispatcher:
    handler: dispatcher.dispatch
    events:
      - sns: arn:xxx

Working output:

$ serverless deploy --verbose
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (10.08 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - geocoder-dev
CloudFormation - UPDATE_IN_PROGRESS - AWS::Lambda::Function - GeocodeLambdaFunction
CloudFormation - UPDATE_COMPLETE - AWS::Lambda::Function - GeocodeLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - GeocodeLambdaPermissionListingseventsdevSNS
CloudFormation - CREATE_IN_PROGRESS - AWS::SNS::Subscription - GeocodeSnsSubscriptionListingseventsdev
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - GeocodeLambdaPermissionListingseventsdevSNS
CloudFormation - CREATE_IN_PROGRESS - AWS::SNS::Subscription - GeocodeSnsSubscriptionListingseventsdev
CloudFormation - CREATE_COMPLETE - AWS::SNS::Subscription - GeocodeSnsSubscriptionListingseventsdev
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Permission - GeocodeLambdaPermissionListingseventsdevSNS
CloudFormation - UPDATE_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - geocoder-dev
CloudFormation - UPDATE_COMPLETE - AWS::CloudFormation::Stack - geocoder-dev
Serverless: Stack update finished...
Service Information
service: geocoder
stage: dev
region: eu-west-2
stack: geocoder-dev
api keys:
  None
endpoints:
  None
functions:
  geocode: geocoder-dev-geocode

Stack Outputs
ServerlessDeploymentBucketName: geocoder-dev-serverlessdeploymentbucket-a9oksuw16rmp
GeocodeLambdaFunctionQualifiedArn: arn:aws:lambda:eu-west-2:xxxxx:function:geocoder-dev-geocode:1

Sorry I’m not sure if this is a bug with the example in the documentation or code?

Thanks a lot for all the hard work on Serverless.

Alex

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Reactions: 11
  • Comments: 21 (2 by maintainers)

Most upvoted comments

We’re being affected by this too. In my research, it appears Serverless either 1) creates both the SNS topic and the Lambda binding, or 2) re-uses an existing SNS topic. It does not update an existing topic with a new binding.

For us, we’re using CI/CD, which means that our deployments cannot update AWS resources automatically. Either we zap the topics manually and have Serverless (re)create & bind them, or we have to manually adjust the Lambda bindings.

Using CI/CD with Serverless sounds like a useful feature, what workarounds have people found?

One direction is for us to use Serverless to create + bind the topic, then we manually import the topic + binding into Terraform. For downstream environments (dev, staging, sales, production), we use Terraform only and tell Serverless to just re-use the TF-created Topics. I’m hoping for a simpler and more durable solution. 😃

And in 2020…

After several hours of trial and error we came up with somthing that worked.

Our use case:

  1. monitor the memory consumption of a lambda function
  2. raise an alarm when > number
  3. Alarm action to send details to SNS Topic
  4. SNS triggers a lambda function (which posts the alarm to slack)

Complete example is not available due to reasons…, but i hope i have included all required parts:

custom:
  alarms:
    topicName: "${env:DEPLOYMENT_NAME}-foobar-alarms"
    topicArn:
      Fn::Join:
        - ":"
        - - "arn:aws:sns"
          - Ref: "AWS::Region"
          - Ref: "AWS::AccountId"
          - "${env:DEPLOYMENT_NAME}-foobar-alarms"

Note, there are more functions, but only this one is subscribed to sns. Beware of the indentation below Two spaces less in front of arn and topicName and it stops working…

functions:
  snsToSlack:
    DependsOn:
      - AlarmTopic
    package:
      exclude:
        - ./**
      include:
        - ./bin/lambdas/all/snsToSlack/main
    handler: bin/lambdas/all/snsToSlack/main
    events:
      - sns:
          arn: ${self:custom.alarms.topicArn}
          topicName: ${self:custom.alarms.topicName}

    AlarmTopic:
      Type: AWS::SNS::Topic
      Properties:
        DisplayName: "Foo Bar Alarms"
        TopicName: ${self:custom.alarms.topicName}
    AlarmTopicPolicy:
      Type: AWS::SNS::TopicPolicy
      DependsOn:
        - AlarmTopic
      Properties:
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Sid: AllowCloudWatchToPublish
              Effect: Allow
              Principal:
                AWS: '*'
              Action: 'sns:Publish'
              Resource:
                Ref: AlarmTopic
        Topics:
          - Ref: AlarmTopic
    FooMemoryMetric:
      Type: "AWS::Logs::MetricFilter"
      Properties:
        FilterPattern: '[ report_label="REPORT", ..., label="Used:", max_memory_used_value, unit="MB" ]'
        LogGroupName:
          Ref: FooLogGroup
        MetricTransformations:
          - MetricName:
              'Fn::Join':
                - '-'
                - - Ref: FooLambdaFunction
                  - MaxMemory
            MetricNamespace: LogMetrics/Lambda
            MetricValue: $max_memory_used_value
    FooMemoryAlarm:
      Type: AWS::CloudWatch::Alarm
      DependsOn:
        - FooLambdaFunction
        - SnsToSlackLambdaFunction
      Properties:
        Namespace: LogMetrics/Lambda
        MetricName:
          'Fn::Join':
            - '-'
            - - Ref: FooLambdaFunction
              - MaxMemory
        AlarmDescription:
          'Fn::Join':
            - ' '
            - - Ref: FooLambdaFunction
              - Memory Alarm
        Threshold: ${self:custom.foo.memoryAlarmThreshold}
        Period: 10
        EvaluationPeriods: 1
        ComparisonOperator: GreaterThanOrEqualToThreshold
        AlarmActions:
          - ${self:custom.alarms.topicArn}
        Statistic: Maximum

For me, i had the same. I entered an existing ARN for an SNS Topic but the deployment never subscribe to this. But i create the Subscription “manual” inside the serverless.yml now:

  Topic:
      Type: AWS::SNS::Topic
      Properties:
        DisplayName: ${self:service}-${opt:stage, self:provider.stage}-CloudWatchAlarmSNS
        Subscription: 
          - Protocol: lambda
            Endpoint: !GetAtt CloudWatchAlarmListenerLambdaFunction.Arn 

  SnsLambdaPermission:        
      Type: AWS::Lambda::Permission
      Properties:
        FunctionName: !GetAtt CloudWatchAlarmListenerLambdaFunction.Arn 
        Action: lambda:InvokeFunction
        SourceArn: !Ref Topic
        Principal: sns.amazonaws.com

But keep in mind, this will only work if you create the SNS Topic at the same time you deploy your Lambda. And you need to create the lambda permission by your own. If you put this into iamRoleStatements it will be a cycle dependency.

Marcel

same, following this one

Having a similar issue although it’s sporadic and haven’t been able to discern what the pattern is to some subscriptions working and some not. What’s strange is that the right directives seem to end up in the Cloudformation template, and AWS knows that I was trying to make the subscription happen. When I use the AWS console to examine a lambda that has a failed subscription, it has this message:

image

Clicking ‘Fix’ then ‘Save’ does indeed fix the subscription. The sporadic nature of this failure is what concerns me the most. I guess we have to add a manual verification step to our deployments.