powertools-lambda-python: SAM CLI Bug: APIGatewayRestResolver and APIGatewayHttpResolver behave differently in local environment
Expected Behaviour
APIGatewayRestResolver
and APIGatewayHttpResolver
should behave similar when StageName: prod
is defined at in SAM
at template.yaml
.
Api:
Type: AWS::Serverless::Api
Properties:
StageName: prod
HttpApi:
Type: AWS::Serverless::HttpApi
Properties:
StageName: prod
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
Events:
HelloPath:
Type: HttpApi
Properties:
RestApiId: !Ref Api
ApiId: !Ref HttpApi
Path: /hello
Method: GET
@app.get("/hello")
@tracer.capture_method
def hello():
In both cases when running sam local start-api
both API gateway instances should try to match /hello
.
Current Behaviour
If StageName: prod
is defined the APIGatewayProxyEventV2
(used by APIGatewayHttpResolver
) will remove len(StageName) + 1
from the request path.
Hence, for this code, APIGatewayRestResolver
will try to match /hello
and APIGatewayHttpResolver
will try to matcho
(/hello - /hell = o
).
If StageName
is not defined (stage name has the default value) the APIGatewayProxyEventV2
will not change the request path making both APIGatewayRestResolver
and APIGatewayHttpResolver
match /hello
.
Faking the stage name is a possible workaround to force APIGatewayHttpResolver
to match /hello
.
def lambda_handler(event: dict, context: LambdaContext) -> dict:
event['requestContext']['stage'] = "$default"
return app.resolve(event, context)
Code snippet
See: https://github.com/tinti/sam-api-httpapi
Possible Solution
I believe something has to be changed at: https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/aws_lambda_powertools/utilities/data_classes/api_gateway_proxy_event.py#L251-L255
If it is not a breaking change I would not remove the stage name at all.
def path(self) -> str:
return self.raw_path
If it is a breaking change maybe create an option
to control whether or not APIGatewayRestResolver
and APIGatewayHttpResolver
should remove the stage name.
Steps to Reproduce
Steps
Create a SAM template application from 1 - AWS Quick Start Templates
then 3 - Hello World Example with Powertools for AWS Lambda
then python3.10
. See: https://github.com/tinti/sam-api-httpapi
Add explicit Api
and HttpApi
resources with StageName: prod
property to the template.
Reference the Api
or HttpApi
resource in the Lambda’s event source.
Start local api with sam local start-api
Test with and without StageName: prod
doing curl -v -X GET --silent localhost:3000/hello
Results
Type | Stage | Expected path | Current path |
---|---|---|---|
Api | $default | /hello | /hello |
Api | prod | /hello | /hello |
HttpApi | $default | /hello | /hello |
HttpApi | prod | /hello | /o |
HttpApi + workaround | prod | /hello | /hello |
Workaround
def lambda_handler(event: dict, context: LambdaContext) -> dict:
event['requestContext']['stage'] = "$default"
return app.resolve(event, context)
Powertools for AWS Lambda (Python) version
latest
AWS Lambda function runtime
3.10
Packaging format used
Lambda Layers
Debugging logs
$ # HttpApi with StageName=prod and @app.get('/hello')
$ sam local start-api
Initializing the lambda functions containers.
Local image is up-to-date
Using local image: public.ecr.aws/lambda/python:3.10-rapid-x86_64.
Mounting <REDACTED>/sam-app/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated, inside runtime container
Containers Initialization is done.
Mounting HelloWorldFunction at http://127.0.0.1:3000/hello [GET]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes
will be reflected instantly/automatically. If you used sam build before running local commands, you will need to re-run sam build for the changes to be
picked up. You only need to restart SAM CLI if you update your AWS SAM template
2023-07-13 06:05:28 WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:3000
2023-07-13 06:05:28 Press CTRL+C to quit
Invoking app.lambda_handler (python3.10)
Reuse the created warm container for Lambda function 'HelloWorldFunction'
Lambda function 'HelloWorldFunction' is already running
START RequestId: b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Version: $LATEST
/var/task/aws_lambda_powertools/package_logger.py:20: UserWarning: POWERTOOLS_DEBUG environment variable is enabled. Setting logging level to DEBUG.
if powertools_debug_is_set():
2023-07-13 09:05:32,222 aws_lambda_powertools.tracing.tracer [DEBUG] Verifying whether Tracing has been disabled
[DEBUG] 2023-07-13T09:05:32.222Z Verifying whether Tracing has been disabled
2023-07-13 09:05:32,518 aws_lambda_powertools.logging.logger [DEBUG] Adding filter in root logger to suppress child logger records to bubble up
[DEBUG] 2023-07-13T09:05:32.518Z Adding filter in root logger to suppress child logger records to bubble up
2023-07-13 09:05:32,518 aws_lambda_powertools.logging.logger [DEBUG] Marking logger PowertoolsHelloWorld as preconfigured
[DEBUG] 2023-07-13T09:05:32.518Z Marking logger PowertoolsHelloWorld as preconfigured
2023-07-13 09:05:32,518 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /hello and methods: GET
[DEBUG] 2023-07-13T09:05:32.518Z Adding route using rule /hello and methods: GET
2023-07-13 09:05:32,519 aws_lambda_powertools.logging.logger [DEBUG] Decorator called with parameters
[DEBUG] 2023-07-13T09:05:32.519Z Decorator called with parameters
2023-07-13 09:05:32,519 aws_lambda_powertools.metrics.base [DEBUG] Decorator called with parameters
[DEBUG] 2023-07-13T09:05:32.519Z Decorator called with parameters
[WARNING] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Subsegment ## lambda_handler discarded due to Lambda worker still initializing
2023-07-13 09:05:32,520 aws_lambda_powertools.tracing.tracer [DEBUG] Calling lambda handler
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Calling lambda handler
{"body":"","cookies":[],"headers":{"Accept":"*/*","Host":"localhost:3000","User-Agent":"curl/8.1.2","X-Forwarded-Port":"3000","X-Forwarded-Proto":"http"},"isBase64Encoded":false,"pathParameters":{},"rawPath":"/hello","rawQueryString":"","requestContext":{"accountId":"123456789012","apiId":"1234567890","domainName":"localhost","domainPrefix":"localhost","http":{"method":"GET","path":"/hello","protocol":"HTTP/1.1","sourceIp":"127.0.0.1","userAgent":"Custom User Agent String"},"requestId":"bb985f80-75a2-47a3-afd0-0467ac443ac9","routeKey":"GET /hello","stage":"prod","time":"13/Jul/2023:09:05:18 +0000","timeEpoch":1689239118},"routeKey":"GET /hello","stageVariables":null,"version":"2.0"}
2023-07-13 09:05:32,520 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Converting event to API Gateway HTTP API contract
2023-07-13 09:05:32,520 aws_lambda_powertools.event_handler.api_gateway [DEBUG] No match found for path o and method GET
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 No match found for path o and method GET
2023-07-13 09:05:32,520 aws_lambda_powertools.metrics.base [DEBUG] Adding cold start metric and function_name dimension
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Adding cold start metric and function_name dimension
2023-07-13 09:05:32,520 aws_lambda_powertools.metrics.base [DEBUG] Adding metric: ColdStart with defaultdict(<class 'list'>, {'Unit': 'Count', 'StorageResolution': 60, 'Value': [1.0]})
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Adding metric: ColdStart with defaultdict(<class 'list'>, {'Unit': 'Count', 'StorageResolution': 60, 'Value': [1.0]})
2023-07-13 09:05:32,520 aws_lambda_powertools.metrics.base [DEBUG] Adding dimension: function_name:HelloWorldFunction
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Adding dimension: function_name:HelloWorldFunction
2023-07-13 09:05:32,520 aws_lambda_powertools.metrics.base [DEBUG] Adding dimension: service:PowertoolsHelloWorld
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Adding dimension: service:PowertoolsHelloWorld
2023-07-13 09:05:32,520 aws_lambda_powertools.metrics.base [DEBUG] {'details': 'Serializing metrics', 'metrics': {'ColdStart': defaultdict(<class 'list'>, {'Unit': 'Count', 'StorageResolution': 60, 'Value': [1.0]})}, 'dimensions': {'function_name': 'HelloWorldFunction', 'service': 'PowertoolsHelloWorld'}}
[DEBUG] 2023-07-13T09:05:32.520Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 {'details': 'Serializing metrics', 'metrics': {'ColdStart': defaultdict(<class 'list'>, {'Unit': 'Count', 'StorageResolution': 60, 'Value': [1.0]})}, 'dimensions': {'function_name': 'HelloWorldFunction', 'service': 'PowertoolsHelloWorld'}}
{"_aws":{"Timestamp":1689239132521,"CloudWatchMetrics":[{"Namespace":"Powertools","Dimensions":[["function_name","service"]],"Metrics":[{"Name":"ColdStart","Unit":"Count"}]}]},"function_name":"HelloWorldFunction","service":"PowertoolsHelloWorld","ColdStart":[1.0]}
/var/task/aws_lambda_powertools/metrics/base.py:418: UserWarning: No application metrics to publish. The cold-start metric may be published if enabled. If application metrics should never be empty, consider using 'raise_on_empty_metrics'
self.flush_metrics(raise_on_empty_metrics=raise_on_empty_metrics)
2023-07-13 09:05:32,521 aws_lambda_powertools.tracing.tracer [DEBUG] Received lambda handler response successfully
[DEBUG] 2023-07-13T09:05:32.521Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Received lambda handler response successfully
2023-07-13 09:05:32,521 aws_lambda_powertools.tracing.tracer [DEBUG] Annotating cold start
[DEBUG] 2023-07-13T09:05:32.521Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Annotating cold start
[WARNING] 2023-07-13T09:05:32.521Z b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 No subsegment to end.
END RequestId: b1a452d8-a8ea-4d4c-8544-19bcd2c8df03
REPORT RequestId: b1a452d8-a8ea-4d4c-8544-19bcd2c8df03 Init Duration: 0.05 ms Duration: 434.74 ms Billed Duration: 435 ms Memory Size: 128 MB Max Memory Used: 128 MB
2023-07-13 06:05:32 127.0.0.1 - - [13/Jul/2023 06:05:32] "GET /hello HTTP/1.1" 404 -
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 1
- Comments: 16 (14 by maintainers)
Hey everyone! Thank you for patience while the SAM CLI team investigated this issue. They confirmed that fixing it would lead to a breaking change, so they’re unable to resolve it.
According to the SAM CLI team, given its complexities for a backwards compatible fix, they’ve marked as a feature request to think of an alternative way to solve this. From today onwards, I’d suggest anyone reading this to subscribe to the SAM CLI issue and add thumbs-up (👍) to help SAM CLI team prioritize it.
Closing this issue.
Hello @tinti! I was able to reproduce the error locally, but I couldn’t when deploying to AWS, which means this is not a bug in a production environment, it’s a bug in the way of emulating this locally.
However, the customer experience is extremely important to us and we know how important the feedback loop is in the software/dev process. I don’t know if it’s really a bug in Powertools or if we need to investigate with the SAM CLI team, but I’ll start a deeper analysis and bring feedback here as soon as I have something relevant.
Thanks for taking the time to create a project to test it out, it’s shortened the path by miles! 🚀
Hello @osjerick! I’m reopening this issue to monitor it and see what we can do to get it done.
The next steps will be: 1 - I’m going to create another issue in the SAM repository with more details, so they can analyze it and verify if it really is a bug and workarounds/solutions. 2 - We will keep in touch with the SAM CLI team.
I’ll update this issue when I have some news.
Thank you for bringing this issue to our attention.
I’ve opened an issue on the upstream project here https://github.com/aws/aws-sam-cli/issues/5579 and I ask you to track it. For the moment we’ll close this issue since the problem doesn’t appear to be caused by Powertools. But please feel free to re-open if you have any additional consideration!
Thank you so much for the well written bug report, it helped tremendously!
Not a problem at runtime.
I notice that the
rawPath
andpath
are not the same when comparing runtime and local.hi @heitorlessa, to be honest I just tested locally. Let me try at runtime and attach the results.
Thanks for the fast reply.
hey @tinti QQ - does this happen at runtime or only when running locally? Reason being is that we’ve seen a different behaviour during emulation and we don’t always account for those (given the plethora of options).
Thank you for the superb bug report - we’ll work on reproducing it now