Zappa: Updating a deployment fails at update_lambda_configuration

Updating a deployment fails at update_lambda_configuration due to a KeyError

Context

I’ve had no problems updating this deployment until today. Python version is 3.8.10.

Expected Behavior

Zappa should complete the update of the deployment successfully.

Actual Behavior

Zappa fails with the following output:

$ zappa update prod
Calling update for stage prod..
Downloading and installing dependencies..
 - pyyaml==5.4.1: Downloading
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 662k/662k [00:00<00:00, 6.57MB/s]
Packaging project as zip.
Uploading tellya2-prod-1631583880.zip (21.9MiB)..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 23.0M/23.0M [00:04<00:00, 4.86MB/s]
Updating Lambda function code..
Updating Lambda function configuration..
Oh no! An error occurred! :(

==============

Traceback (most recent call last):
  File "/Users/mark/.virtualenvs/tellya2_project/lib/python3.8/site-packages/zappa/cli.py", line 3422, in handle
    sys.exit(cli.handle())
  File "/Users/mark/.virtualenvs/tellya2_project/lib/python3.8/site-packages/zappa/cli.py", line 588, in handle
    self.dispatch_command(self.command, stage)
  File "/Users/mark/.virtualenvs/tellya2_project/lib/python3.8/site-packages/zappa/cli.py", line 641, in dispatch_command
    self.update(
  File "/Users/mark/.virtualenvs/tellya2_project/lib/python3.8/site-packages/zappa/cli.py", line 1165, in update
    self.lambda_arn = self.zappa.update_lambda_configuration(
  File "/Users/mark/.virtualenvs/tellya2_project/lib/python3.8/site-packages/zappa/core.py", line 1395, in update_lambda_configuration
    if lambda_aws_config["PackageType"] != "Image":
KeyError: 'PackageType'

==============

Need help? Found a bug? Let us know! :D
File bug reports on GitHub here: https://github.com/Zappa/Zappa
And join our Slack channel here: https://zappateam.slack.com
Love!,
 ~ Team Zappa!

Possible Fix

None that I’m aware of.

Steps to Reproduce

Unfortunately this is a private code repository so I cannot provide any links.

Your Environment

  • Zappa version used: 0.53.0
  • Operating System and Python version: MacOS 11.5.1 - Python 3.8.10
  • The output of pip freeze: argcomplete==1.12.3 asgiref==3.3.4 boto3==1.17.97 botocore==1.20.97 certifi==2021.5.30 cfn-flip==1.2.3 chardet==4.0.0 click==8.0.1 Django==3.2.4 django-appconf==1.0.4 django-compressor @ git+https://github.com/django-compressor/django-compressor.git@f533dc6dde9ed90626382d78f3eb37b84d848027 django-formtools==2.3 django-libsass==0.9 django-otp==1.0.6 django-phonenumber-field==5.2.0 django-storages==1.11.1 django-tenants==3.3.1 django-two-factor-auth==1.13.1 durationpy==0.5 future==0.18.2 hjson==3.0.2 idna==2.10 jmespath==0.10.0 kappa==0.6.0 libsass==0.21.0 pbr==5.6.0 pep517==0.10.0 phonenumbers==8.12.25 pip-tools==6.1.0 placebo==0.9.0 psycopg2==2.9.1 psycopg2-binary==2.9.1 PyJWT==1.7.1 python-dateutil==2.8.1 python-slugify==5.0.2 pytz==2021.1 PyYAML==5.4.1 qrcode==6.1 rcssmin==1.0.6 requests==2.25.1 rjsmin==1.1.0 s3transfer==0.4.2 six==1.15.0 sqlparse==0.4.1 stevedore==3.3.0 text-unidecode==1.3 toml==0.10.2 tqdm==4.61.1 troposphere==2.7.0 twilio==6.61.0 urllib3==1.26.5 Werkzeug==0.16.1 wsgi-request-logger==0.4.6 zappa==0.53.0 zappa-django-utils==0.4.1
  • Your zappa_settings.json:
{
    "dev": {
        "aws_region": "ap-northeast-1",
        "django_settings": "config.settings.dev",
        "profile_name": "default",
        "project_name": "tellya2",
        "runtime": "python3.8",
        "s3_bucket": "tellya2-dev",
        "keep_warm": false,
        "vpc_config": {
            "SubnetIds": ["subnet-085938da970945e07"],
            "SecurityGroupIds": ["sg-0353477a152c17539"]
        },
        "layers": ["arn:aws:lambda:ap-northeast-1:898466741470:layer:psycopg2-py38:1"],
        "delete_local_zip": true,
        "exclude": ["static", "psycopg2", "*.pyc", "pip", "pip-tools", "setuptools", "*.dist-info", "*.so"],
        "events": [{
            "function": "tellya2.lambda_functions.check_messages", 
            "expression": "rate(1 minute)"
        }]
    },
    "prod": {
        "aws_region": "ap-northeast-1",
        "django_settings": "config.settings.prod",
        "profile_name": "default",
        "project_name": "tellya2",
        "runtime": "python3.8",
        "s3_bucket": "tellya2-prod",
        "keep_warm": true,
        "vpc_config": {
            "SubnetIds": ["subnet-0042d4fa27f9253ae", "subnet-0768c029b7e2d32c3"],
            "SecurityGroupIds": ["sg-446b8d00"]
        },
        "layers": ["arn:aws:lambda:ap-northeast-1:898466741470:layer:psycopg2-py38:1"],
        "delete_local_zip": true,
        "exclude": ["static", "psycopg2", "pip*", "pip-tools*", "setuptools", "*.pyc", "*.dist-info", "*.so"],
        "events": [{
            "function": "tellya2.lambda_functions.check_messages", 
            "expression": "rate(1 minute)"
        }],
        "certificate_arn": "arn:aws:acm:us-east-1:214849182730:certificate/19237cc2-0571-4045-bc10-face4e581f3d",
        "domain": "[redacted]"
    }
}

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 4
  • Comments: 25 (2 by maintainers)

Commits related to this issue

Most upvoted comments

Resolved as of v0.54.0 via #992

Is there any plan for making a release with the latest fix commit?

I see that Zappa has been officially suddenly deprecated without warning in the most recent commits. That’s a real shame.

I opened a PR to fix this here: #1042

Details are in the PR description, but the TL;DR is that the code currently assumes that the Lambda function is fully initialized at this point, but if it isn’t, the response will not contain the PackageType key. AWS (at least us-east-1) currently seems to be suffering some kind of performance degradation where the Lambda function is frequently not initialized in time.

@ohadbenita This is happening to me as well in us-east-1, can you list out which regions you’re noticing this issue with? This is an ongoing issue for me on version 0.54.1 on a fresh install.

I was facing same issue, I uninstalled and reinstalled the updated package. Problem is resolved now.

Side note - I tried to join the Slack workspace given in the output, but it does not actually seem possible to join it. Perhaps that message should be removed?