Zappa: SciPy module not found

Context

I have these settings:

"slim_handler": true,
"use_precompiled_packages": true

My application requires scipy, and it is installed in the virtual environment. When I deploy my application I see this:

Calling deploy for stage dev..
Downloading and installing dependencies..
 - scipy==0.19.0: Using locally cached manylinux wheel
File is not a zip file

Expected Behavior

After zappa deploy dev it says “Deployment complete” in green letters, so I would expect the application to run correctly (e.g. answer curl requests correctly).

Actual Behavior

If I send a curl request, I get

HTTP/1.1 500 Internal Server Error

and the error message ends with

"TypeError: ‘NoneType’ object is not callable\n"]

If I run zappa tail, I see this:

[1505391412036] No module named 'scipy': ModuleNotFoundError
Traceback (most recent call last):
  File "/var/task/handler.py", line 505, in lambda_handler
  return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 239, in lambda_handler
  handler = cls()
  File "/var/task/handler.py", line 131, in __init__
  self.app_module = importlib.import_module(self.settings.APP_MODULE)
  File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
  return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/tmp/channel-recomme/channel_recommender/webapp/webapp.py", line 24, in <module>
  from channel_recommender.recommender.loader import service_loader
  File "/tmp/channel-recomme/channel_recommender/recommender/loader.py", line 7, in <module>
  from sklearn.externals import joblib
  File "/tmp/channel-recomme/sklearn/__init__.py", line 134, in <module>
  from .base import clone
  File "/tmp/channel-recomme/sklearn/base.py", line 10, in <module>
  from scipy import sparse
ModuleNotFoundError: No module named 'scipy'

If I undeploy, disable using precompiled packages, i.e. I set this value: "use_precompiled_packages": false and then redeploy, then everything works as expected, i.e. the application answers curl correctly.

This is a bit strange, because initially (two weeks ago more or less) my application required scipy (I used scikit-learn, which required numpy and scipy) and this issue was not present (i.e. I could deploy the application without including use_precompiled_packages in my settings (i.e. it got the default value which is true).

However, a newer version of my application requires both scikit-learn and pandas. Therefore, I added pandas to the virtual environment. After that, the next time that I deployed I found the issue that I described.

I have tried uninstalling the packages involved and installing them again, but the issue is still there. I have tried undeploying the application and deploying it again, without any change. The only thing that seems to work is to set use_precompiled_packages to false, but as I mentioned earlier, this was initially not necessary.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 1
  • Comments: 21 (12 by maintainers)

Most upvoted comments

Ran into the same issue, spent a while figuring out what was wrong !

Thanks a lot @alfonsomhc for the "use_precompiled_packages": false, this solved it for me.