Zappa: how do you debug errors

I just deployed a Django app but nothing works (deploy went fine). running check management command returns:

populate() isn't reentrant: RuntimeError
Traceback (most recent call last):
  File "/var/task/handler.py", line 460, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 208, in lambda_handler
    raise ex
RuntimeError: populate() isn't reentrant

And any request to the app outputs:

{
    "message": "An uncaught exception happened while servicing this request.", 
    "traceback": [
        "Traceback (most recent call last):\n", 
        "  File \"/var/task/handler.py\", line 386, in handler\n    response = Response.from_app(self.wsgi_app, environ)\n", 
        "  File \"/private/var/folders/41/q6bnwcwn07lg0_4zdwfsq2gc0000gn/T/pip-build-4PP29N/Werkzeug/werkzeug/wrappers.py\", line 865, in from_app\n", 
        "  File \"/private/var/folders/41/q6bnwcwn07lg0_4zdwfsq2gc0000gn/T/pip-build-4PP29N/Werkzeug/werkzeug/test.py\", line 871, in run_wsgi_app\n", 
        "TypeError: 'NoneType' object is not callable\n"
    ]
}

Unless I missed something these errors are completely useless; is there any way you can find more information about what is not working with a deployment?

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 19 (8 by maintainers)

Most upvoted comments

Ok, I found the solution for this issue in my case.

I had a django project layout like this:

── README.md
├── lambda-requirements.txt
├── requirements.txt
├── static
├── zappa_settings.json
└── projectname
    ├── appone
    │   ├── __init__.py
    │   ├── admin.py
    │   ├── apps.py
    │   ├── fixtures
    │   ├── migrations
    │   ├── models.py
    │   ├── tests.py
    │   └── views.py
    ├── manage.py
    ├── apptwo
    │   ├── __init__.py
    │   ├── admin.py
    │   ├── apps.py
    │   ├── models.py
    │   ├── templates
    │   ├── tests.py
    │   ├── urls.py
    │   └── views.py
    ├── projectname
    │   ├── __init__.py
    │   ├── db.sqlite3
    │   ├── settings
    │   │   ├── __init__.py
    │   │   ├── base.py
    │   │   ├── dev.py
    │   │   └── local.py
    │   ├── urls.py
    │   └── wsgi.py
    ├── templates

So in this layout zappa_settings.json was at the top level, and I need to set my django_settings like this:

{
   "dev": {
        ...
        "django_settings": "projectname.projectname.settings.dev",
        ...
    }
}

While this allowed me to seemingly get things working I ran into this populate() error when attempting to run, zappa manage dev migrate initially. Digging in further I found that by using zappa tail my set application, appone defined in INSTALLED_APPS was not being found. (even after renaming my app, thinking that I had some potential naming conflict…)

Finally, moving the zappa_settings.json configuration file from the project root directory, ./zappa_settings.json, to within the initial project directory, ./projectname/zappa_settings.json, resolved the issue, and I was able to run the initial zappa manage dev migrate command successfully!

Btw, this task have more life that I expected. Errors are getting lost somewhere, and we get this error. It can be safely replaced to message “Were was an exception, look for it in zappa tail output.”

As I see when something bad happens on lambda/zappa side, original exception gets eaten by this message, and no debug information shown to user.

May be worth forking werkzeug and make more detailed explanation about what happened in such cases.

Today I got this error when newrelic package wasn’t pushed to zappa (fixed by ln -s newrelic-2.70.0.51/newrelic/ ./newrelic); and some guy got it because django wasn’t able to make local log file, and I can’t even imagine how many people have it because of some environment problems, unrelated to zappa, and never see the source exception.