aws-sam-cli: sam local start-api fails to move /var/rapid/aws-lambda-rie-x86_64 in docker image for serverless function

@chrisrhymes I managed to fix this by executing the following commands

$ brew uninstall aws-sam-cli
$ brew install rust
$ brew install aws/tap/aws-sam-cli

Hi @mjvirt I’m also having this same issue when using an M1 Mac running macOS 13.1.

I changed the template.yml Architecture to arm64, reran sam build and then sam local invoke --no-event worked for me.

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs18.x
      Architectures:
        - arm64 # Update this line for it to work locally

Hi @chrisrhymes and others, maybe I wasn’t clear in my original message above, but this last line works:

I did find this and have managed to get it working for SAM: https://github.com/tonistiigi/binfmt (other CPU architecture emulation on arm64/M1)

In other words you can emulate x86_64 on arm64 - and it works fine with SAM/CDK - at least insofar as I’ve tested it (just use docker run --privileged --rm tonistiigi/binfmt --install all to install the available platform emulators; or you can be more picky…).

I have just run into this issue and it affects sam local invoke etc.

TL;DR: Lambda functions default to Linux/x86_64 architecture even when run locally with SAM. On Linux for arm/aarch(64) only the Docker Engine/CE is available to install. Unlike for Docker Desktop, CE doesn’t include QEMU emulation configuration for other platforms (specifically Linux/x86_64). Third-party binfmt is needed. This on very light testing appears to resolve the issue for SAM.

Longer version:

docker ps -a and info shows the command that fails:

"mv /var/rapid/aws-lambda-rie-x86_64 /var/rapid/aws-lambda-rie && chmod +x /var/rapid/aws-lambda-rie"

Additional environment details:

OS: Fedora Linux 36 (ARM64) - Parallels VM on Macbook M1
sam --version: SAM CLI, version 1.56.0
AWS region: eu-central-1

It is fairly easy to reproduce. Just create the hello-world sam app via $ sam init in an empty directory: 1 - AWS Quick Start Templates 1 - Hello World Example

$ sam local invoke HelloWorldFunction
Invoking app.lambda_handler (python3.9)
Image was not found.
Removing rapid images for repo public.ecr.aws/sam/emulation-python3.9
Building image...................
Failed to build Docker Image
NoneType: None
Error: Error building docker image: The command '/bin/sh -c mv /var/rapid/aws-lambda-rie-x86_64 /var/rapid/aws-lambda-rie && chmod +x /var/rapid/aws-lambda-rie' returned a non-zero code: 1

The problem is due to a platform mismatch - attempting to run x86_64 images on arm64 architecture. Ordinarily with Docker Desktop on x86_64 it’s possible to emulate other architectures such as arm64.

Per the AWS SAM document here https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install-linux.html it states:

On Linux, to build and run Lambda functions with a different instruction set architecture than your host machine, you must take additional steps to configure Docker. For example, to run arm64 functions on an x86_64 machine, you can run the following command to configure the Docker daemon: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes.

However, multiarch/qemu-user-static only supports the x86_64 host architecture per https://github.com/multiarch/qemu-user-static/.

I did find this and have managed to get it working for SAM: https://github.com/tonistiigi/binfmt.

Thanks @ixolt

The new version of AWS SAM CLI - (ﾉ◕ヮ◕)ﾉ*:･ﾟ✧ The problem Fixed

$ brew uninstall aws-sam-cli
$ brew install rust
$ brew tap aws/tap
$ brew install aws-sam-cli

Yes, I’ll give it a look when I’ll have time