Zappa: SciPy module not found
Context
I have these settings:
"slim_handler": true,
"use_precompiled_packages": true
My application requires scipy, and it is installed in the virtual environment. When I deploy my application I see this:
Calling deploy for stage dev..
Downloading and installing dependencies..
- scipy==0.19.0: Using locally cached manylinux wheel
File is not a zip file
Expected Behavior
After zappa deploy dev it says “Deployment complete” in green letters, so I would expect the application to run correctly (e.g. answer curl requests correctly).
Actual Behavior
If I send a curl request, I get
HTTP/1.1 500 Internal Server Error
and the error message ends with
"TypeError: ‘NoneType’ object is not callable\n"]
If I run zappa tail, I see this:
[1505391412036] No module named 'scipy': ModuleNotFoundError
Traceback (most recent call last):
File "/var/task/handler.py", line 505, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 239, in lambda_handler
handler = cls()
File "/var/task/handler.py", line 131, in __init__
self.app_module = importlib.import_module(self.settings.APP_MODULE)
File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "/tmp/channel-recomme/channel_recommender/webapp/webapp.py", line 24, in <module>
from channel_recommender.recommender.loader import service_loader
File "/tmp/channel-recomme/channel_recommender/recommender/loader.py", line 7, in <module>
from sklearn.externals import joblib
File "/tmp/channel-recomme/sklearn/__init__.py", line 134, in <module>
from .base import clone
File "/tmp/channel-recomme/sklearn/base.py", line 10, in <module>
from scipy import sparse
ModuleNotFoundError: No module named 'scipy'
If I undeploy, disable using precompiled packages, i.e. I set this value:
"use_precompiled_packages": false
and then redeploy, then everything works as expected, i.e. the application answers curl correctly.
This is a bit strange, because initially (two weeks ago more or less) my application required scipy (I used scikit-learn, which required numpy and scipy) and this issue was not present (i.e. I could deploy the application without including use_precompiled_packages in my settings (i.e. it got the default value which is true).
However, a newer version of my application requires both scikit-learn and pandas. Therefore, I added pandas to the virtual environment. After that, the next time that I deployed I found the issue that I described.
I have tried uninstalling the packages involved and installing them again, but the issue is still there. I have tried undeploying the application and deploying it again, without any change. The only thing that seems to work is to set use_precompiled_packages to false, but as I mentioned earlier, this was initially not necessary.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 1
- Comments: 21 (12 by maintainers)
Ran into the same issue, spent a while figuring out what was wrong !
Thanks a lot @alfonsomhc for the
"use_precompiled_packages": false, this solved it for me.