chalice: chalice deployment package missing libraries
When my Bitbucket pipeline runs chalice deploy --stage dev
, the deployment works just fine. When I run chalice deploy --stage dev
from my command line, the package deployed to AWS is missing most libraries… so I get errors like Unable to import module 'app': No module named 'phpserialize'
when the Lambda is executed.
I’ve verified that the deployment packages differ by running deployments, downloading the deployment packages from AWS (using the Lambda console), and diff’ing the contents of the .zip files.
Clearly there’s some difference between my personal environment and the Bitbucket pipeline environment that causes a difference in how the deployment .zip is built, but I can’t figure it out. Here’s what I’ve tried:
- Changing the commands to/from
chalice deploy
&&chalice deploy --stage dev
. No affect. - Rebuilding my vendor dependencies: In the past I’ve been able to ‘fix’ deployments missing modules by manually rebuilding the wheels in the
vendor
directory, but this hasn’t worked in the past few days:pip download x && pip wheel x-y.y.y.tar.gz && rm *.gz
- python/pip versions: Bitbucket was using python 3.6.3 & pip 9.0.1 and my personal machine was python 3.6.5 & pip 9.0.3. I changed my versions 3.6.3/9.0.1 and my deployment packages were still missing libraries. I changed the Bitbucket pipeline to 3.6.5/10.0.1 and the deployment was still perfect.
- I’ve also reviewed issues: #106, #155, #189
Anyone else seen this problem? It’s plaguing my office here and some of us have turned to pushing all changes to Bitbucket because we can’t trust a local chalice deploy
to work.
Bitbucket OS: Linux 44faaabc-f42c-4803-993e-f72fa3365e1e 4.14.48-coreos-r2 #1 SMP Thu Jun 14 08:23:03 UTC 2018 x86_64 GNU/Linux
Laptop OS: Darwin Macbook.localdomain 17.6.0 Darwin Kernel Version 17.6.0: Tue May 8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64 x86_64
Here’s my requirements.txt
file:
chalice==1.3.0
phpserialize==1.3
yoyo-migrations==5.1.5
Here’re the contents of my vendor
directory:
MarkupSafe-1.0-cp36-cp36m-macosx_10_13_x86_64.whl idna-2.7-py2.py3-none-any.whl
PyYAML-3.13-cp36-cp36m-macosx_10_13_x86_64.whl iniherit-0.3.9-py3-none-any.whl
argh-0.26.2-py2.py3-none-any.whl jmespath-0.9.3-py2.py3-none-any.whl
argparse-1.4.0-py2.py3-none-any.whl pathtools-0.1.2-py3-none-any.whl
asn1crypto-0.24.0-py2.py3-none-any.whl phpserialize-1.3-py3-none-any.whl
bidict-0.17.2-py3-none-any.whl pycparser-2.18-py2.py3-none-any.whl
boto3-1.7.42-py2.py3-none-any.whl python_dateutil-2.7.3-py2.py3-none-any.whl
botocore-1.10.42-py2.py3-none-any.whl pytz-2018.4-py2.py3-none-any.whl
cffi-1.11.5-cp36-cp36m-macosx_10_6_intel.whl s3transfer-0.1.13-py2.py3-none-any.whl
ciso8601-2.0.1.tar.gz schemapi-0.3.0-py3-none-any.whl
credstash-1.14.0-py3-none-any.whl simplejson-3.15.0-cp36-cp36m-macosx_10_13_x86_64.whl
cryptography-2.0.3-cp36-cp36m-macosx_10_6_intel.whl six-1.11.0-py2.py3-none-any.whl
dateutils-0.6.6-py3-none-any.whl typing-3.5.3.0-py3-none-any.whl
docutils-0.14-py3-none-any.whl watchdog-0.8.3-cp36-cp36m-macosx_10_13_x86_64.whl
expiringdict-1.1.4-py3-none-any.whl
Here’s a sanitized copy of my Bitbucket pipeline file:
image: python:3.6.5
pipelines:
branches:
develop:
- step:
script:
- export AWS_ACCESS_KEY_ID=$DEV_AWS_ACCESS_KEY_ID
- export AWS_SECRET_ACCESS_KEY=$DEV_AWS_SECRET_ACCESS_KEY
- pip install -r requirements.txt
- pip install pytest
- python -m pytest tests/
- chalice deploy --stage dev
deployment: staging
Here’s a diff of the deployment package (Bitbucket deployed vs. manually deployed):
Only in notifications-poller-dev-Bitbucket: MarkupSafe-1.0.dist-info
Only in notifications-poller-dev-Bitbucket: PyYAML-3.13.dist-info
Only in notifications-poller-dev-Bitbucket: _yaml.cpython-36m-x86_64-linux-gnu.so
Only in notifications-poller-dev-Bitbucket: bidict
Only in notifications-poller-dev-Bitbucket: bidict-0.17.2.dist-info
Only in notifications-poller-dev-Bitbucket: ciso8601-2.0.1.dist-info
Only in notifications-poller-dev-Bitbucket: ciso8601.cpython-36m-x86_64-linux-gnu.so
Only in notifications-poller-dev-Bitbucket: dateutils
Only in notifications-poller-dev-Bitbucket: dateutils-0.6.6.dist-info
Only in notifications-poller-dev-Bitbucket: iniherit
Only in notifications-poller-dev-Bitbucket: iniherit-0.3.9.dist-info
Only in notifications-poller-dev-Bitbucket: markupsafe
Only in notifications-poller-dev-Bitbucket: pathtools
Only in notifications-poller-dev-Bitbucket: pathtools-0.1.2.dist-info
Only in notifications-poller-dev-Bitbucket: phpserialize-1.3.dist-info
Only in notifications-poller-dev-Bitbucket: phpserialize.py
Only in notifications-poller-dev-Bitbucket: pycparser
Only in notifications-poller-dev-Bitbucket: pycparser-2.18.dist-info
Only in notifications-poller-dev-Bitbucket: schemapi
Only in notifications-poller-dev-Bitbucket: schemapi-0.3.0.dist-info
Only in notifications-poller-dev-Bitbucket: watchdog
Only in notifications-poller-dev-Bitbucket: watchdog-0.8.3.dist-info
Only in notifications-poller-dev-Bitbucket: yaml
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 16 (6 by maintainers)
I’m currently running into an issue wher Chalice wont deploy local code correctly and is actively skipping the packaging of code we depend on that is work we did ourselves that is a local module that is not installed via pip because its just a local library we created and consume; could this be related?