snowflake-connector-python: AWS lambda docker container cannot resolve dependencies

Please answer these questions before submitting your issue. Thanks!

1. What version of Python are you using (python --version)? Python 3.7.5

2. What operating system and processor architecture are you using (python -c 'import platform; print(platform.platform())')? Linux-5.0.0-37-generic-x86_64-with-Ubuntu-18.04-bionic

3. What are the component versions in the environment (pip list)? Package Version

---

asn1crypto 1.2.0
awscli 1.16.306
azure-common 1.1.24
azure-storage-blob 2.1.0
azure-storage-common 2.1.0
boto3 1.10.43
botocore 1.13.43
certifi 2019.11.28 cffi 1.13.2
chardet 3.0.4
colorama 0.4.1
cryptography 2.8
docutils 0.15.2
future 0.18.2
idna 2.8
ijson 2.5.1
jmespath 0.9.4
numpy 1.17.4
oscrypto 1.1.0
pandas 0.25.3
pip 19.3.1
pyasn1 0.4.8
pycparser 2.19
pycryptodomex 3.9.4
PyJWT 1.7.1
pyOpenSSL 19.1.0
python-dateutil 2.8.0
pytz 2019.3
PyYAML 5.1.2
requests 2.22.0
rsa 3.4.2
s3transfer 0.2.1
setuptools 41.6.0
six 1.13.0
snowflake-connector-python 2.1.2
snowflake-sqlalchemy 1.1.17
SQLAlchemy 1.3.12
urllib3 1.25.7
wheel 0.33.6

1. What did you do? If possible, provide a recipe for reproducing the error. A complete runnable program is good. Used AWS SAM v0.38.0, added snowflake-connector-python to helloworld sam example’s requirements.txt file.

Try to build the example in an AWS lambda container with sam build -u

1. What did you expect to see? Build succeeded

2. What did you see instead? anton@anton-Virtual-Machine:~/aws/samexample/sam-app$ sam build -u Starting Build inside a container Building resource ‘HelloWorldFunction’

Fetching lambci/lambda:build-python3.7 Docker container image… Mounting /home/anton/aws/samexample/sam-app/hello_world as /tmp/samcli/source:ro,delegated inside runtime container

Build Failed Running PythonPipBuilder:ResolveDependencies Error: PythonPipBuilder:ResolveDependencies - {snowflake-connector-python==2.1.2(wheel)}

1. Can you set logging to DEBUG and collect the logs? With sam build debug:

anton@anton-Virtual-Machine:~/aws/samexample/sam-app$ sam build -u --debug Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics ‘build’ command is called Starting Build inside a container No Parameters detected in the template 2 resources found in the template Found Serverless function with name=‘HelloWorldFunction’ and CodeUri=‘hello_world/’ Building resource ‘HelloWorldFunction’

Fetching lambci/lambda:build-python3.7 Docker container image… Mounting /home/anton/aws/samexample/sam-app/hello_world as /tmp/samcli/source:ro,delegated inside runtime container Using the request object from command line argument Loading workflow module ‘aws_lambda_builders.workflows’ Registering workflow ‘PythonPipBuilder’ with capability ‘Capability(language=‘python’, dependency_manager=‘pip’, application_framework=None)’ Registering workflow ‘NodejsNpmBuilder’ with capability ‘Capability(language=‘nodejs’, dependency_manager=‘npm’, application_framework=None)’ Registering workflow ‘RubyBundlerBuilder’ with capability ‘Capability(language=‘ruby’, dependency_manager=‘bundler’, application_framework=None)’ Registering workflow ‘GoDepBuilder’ with capability ‘Capability(language=‘go’, dependency_manager=‘dep’, application_framework=None)’ Registering workflow ‘GoModulesBuilder’ with capability ‘Capability(language=‘go’, dependency_manager=‘modules’, application_framework=None)’ Registering workflow ‘JavaGradleWorkflow’ with capability ‘Capability(language=‘java’, dependency_manager=‘gradle’, application_framework=None)’ Registering workflow ‘JavaMavenWorkflow’ with capability ‘Capability(language=‘java’, dependency_manager=‘maven’, application_framework=None)’ Registering workflow ‘DotnetCliPackageBuilder’ with capability ‘Capability(language=‘dotnet’, dependency_manager=‘cli-package’, application_framework=None)’ Found workflow ‘PythonPipBuilder’ to support capabilities ‘Capability(language=‘python’, dependency_manager=‘pip’, application_framework=None)’ Running workflow ‘PythonPipBuilder’ Running PythonPipBuilder:ResolveDependencies calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch Full dependency closure: {docutils==0.15.2(wheel), azure-storage-common==2.1.0(wheel), oscrypto==1.1.0(wheel), pandas==0.25.3(wheel), cffi==1.13.2(wheel), chardet==3.0.4(wheel), ijson==2.5.1(wheel), azure-common==1.1.24(wheel), idna==2.8(wheel), urllib3==1.25.7(wheel), pycryptodomex==3.9.4(wheel), python-dateutil==2.8.1(wheel), s3transfer==0.2.1(wheel), certifi==2019.11.28(wheel), cryptography==2.8(wheel), asn1crypto==1.2.0(wheel), pytz==2019.3(wheel), six==1.13.0(wheel), sqlalchemy==1.3.12(sdist), pyopenssl==19.1.0(wheel), pycparser==2.19(sdist), snowflake-connector-python==2.1.2(wheel), azure-storage-blob==2.1.0(wheel), botocore==1.13.43(wheel), jmespath==0.9.4(wheel), boto3==1.10.43(wheel), requests==2.22.0(wheel), future==0.18.2(sdist), numpy==1.17.4(wheel), pyjwt==1.7.1(wheel)} initial compatible: {docutils==0.15.2(wheel), azure-storage-common==2.1.0(wheel), oscrypto==1.1.0(wheel), pandas==0.25.3(wheel), cffi==1.13.2(wheel), chardet==3.0.4(wheel), ijson==2.5.1(wheel), azure-common==1.1.24(wheel), idna==2.8(wheel), urllib3==1.25.7(wheel), pycryptodomex==3.9.4(wheel), python-dateutil==2.8.1(wheel), s3transfer==0.2.1(wheel), certifi==2019.11.28(wheel), asn1crypto==1.2.0(wheel), pytz==2019.3(wheel), six==1.13.0(wheel), pyopenssl==19.1.0(wheel), azure-storage-blob==2.1.0(wheel), botocore==1.13.43(wheel), jmespath==0.9.4(wheel), boto3==1.10.43(wheel), requests==2.22.0(wheel), numpy==1.17.4(wheel), pyjwt==1.7.1(wheel)} initial incompatible: {cryptography==2.8(wheel), future==0.18.2(sdist), snowflake-connector-python==2.1.2(wheel), pycparser==2.19(sdist), sqlalchemy==1.3.12(sdist)} Downloading missing wheels: {cryptography==2.8(wheel), future==0.18.2(sdist), snowflake-connector-python==2.1.2(wheel), pycparser==2.19(sdist), sqlalchemy==1.3.12(sdist)} calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch cryptography==2.8 calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch future==0.18.2 calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch snowflake-connector-python==2.1.2 calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch pycparser==2.19 calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch sqlalchemy==1.3.12 compatible wheels after second download pass: {docutils==0.15.2(wheel), azure-storage-common==2.1.0(wheel), oscrypto==1.1.0(wheel), pandas==0.25.3(wheel), cffi==1.13.2(wheel), chardet==3.0.4(wheel), ijson==2.5.1(wheel), azure-common==1.1.24(wheel), idna==2.8(wheel), urllib3==1.25.7(wheel), pycryptodomex==3.9.4(wheel), python-dateutil==2.8.1(wheel), s3transfer==0.2.1(wheel), certifi==2019.11.28(wheel), cryptography==2.8(wheel), asn1crypto==1.2.0(wheel), pytz==2019.3(wheel), six==1.13.0(wheel), pyopenssl==19.1.0(wheel), azure-storage-blob==2.1.0(wheel), botocore==1.13.43(wheel), jmespath==0.9.4(wheel), boto3==1.10.43(wheel), requests==2.22.0(wheel), numpy==1.17.4(wheel), pyjwt==1.7.1(wheel)} Build missing wheels from sdists (C compiling True): {pycparser==2.19(sdist), future==0.18.2(sdist), sqlalchemy==1.3.12(sdist)} calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/pycparser-2.19.tar.gz calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/future-0.18.2.tar.gz calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/SQLAlchemy-1.3.12.tar.gz compatible after building wheels (no C compiling): {docutils==0.15.2(wheel), azure-storage-common==2.1.0(wheel), oscrypto==1.1.0(wheel), pandas==0.25.3(wheel), cffi==1.13.2(wheel), chardet==3.0.4(wheel), ijson==2.5.1(wheel), azure-common==1.1.24(wheel), idna==2.8(wheel), urllib3==1.25.7(wheel), pycryptodomex==3.9.4(wheel), python-dateutil==2.8.1(wheel), s3transfer==0.2.1(wheel), certifi==2019.11.28(wheel), cryptography==2.8(wheel), asn1crypto==1.2.0(wheel), pytz==2019.3(wheel), six==1.13.0(wheel), sqlalchemy==1.3.12(wheel), pyopenssl==19.1.0(wheel), pycparser==2.19(wheel), azure-storage-blob==2.1.0(wheel), botocore==1.13.43(wheel), jmespath==0.9.4(wheel), boto3==1.10.43(wheel), requests==2.22.0(wheel), future==0.18.2(wheel), numpy==1.17.4(wheel), pyjwt==1.7.1(wheel)} Build missing wheels from sdists (C compiling False): set() compatible after building wheels (C compiling): {docutils==0.15.2(wheel), azure-storage-common==2.1.0(wheel), oscrypto==1.1.0(wheel), pandas==0.25.3(wheel), cffi==1.13.2(wheel), chardet==3.0.4(wheel), ijson==2.5.1(wheel), azure-common==1.1.24(wheel), idna==2.8(wheel), urllib3==1.25.7(wheel), pycryptodomex==3.9.4(wheel), python-dateutil==2.8.1(wheel), s3transfer==0.2.1(wheel), certifi==2019.11.28(wheel), cryptography==2.8(wheel), asn1crypto==1.2.0(wheel), pytz==2019.3(wheel), six==1.13.0(wheel), sqlalchemy==1.3.12(wheel), pyopenssl==19.1.0(wheel), pycparser==2.19(wheel), azure-storage-blob==2.1.0(wheel), botocore==1.13.43(wheel), jmespath==0.9.4(wheel), boto3==1.10.43(wheel), requests==2.22.0(wheel), future==0.18.2(wheel), numpy==1.17.4(wheel), pyjwt==1.7.1(wheel)} Final compatible: {s3transfer==0.2.1(wheel), certifi==2019.11.28(wheel), cryptography==2.8(wheel), docutils==0.15.2(wheel), asn1crypto==1.2.0(wheel), pytz==2019.3(wheel), six==1.13.0(wheel), azure-storage-common==2.1.0(wheel), oscrypto==1.1.0(wheel), sqlalchemy==1.3.12(wheel), pyopenssl==19.1.0(wheel), pandas==0.25.3(wheel), cffi==1.13.2(wheel), pycparser==2.19(wheel), chardet==3.0.4(wheel), azure-storage-blob==2.1.0(wheel), ijson==2.5.1(wheel), azure-common==1.1.24(wheel), botocore==1.13.43(wheel), jmespath==0.9.4(wheel), idna==2.8(wheel), boto3==1.10.43(wheel), requests==2.22.0(wheel), urllib3==1.25.7(wheel), future==0.18.2(wheel), pycryptodomex==3.9.4(wheel), numpy==1.17.4(wheel), python-dateutil==2.8.1(wheel), pyjwt==1.7.1(wheel)} Final incompatible: {snowflake-connector-python==2.1.2(wheel), cryptography==2.8(wheel)} Final missing wheels: {snowflake-connector-python==2.1.2(wheel)} PythonPipBuilder:ResolveDependencies failed Traceback (most recent call last): File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py”, line 39, in execute requirements_path=self.manifest_path File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py”, line 144, in build_dependencies requirements_path, artifacts_dir_path, scratch_dir_path) File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py”, line 213, in build_site_packages raise MissingDependencyError(packages_without_wheels) aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {snowflake-connector-python==2.1.2(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py”, line 248, in run action.execute() File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py”, line 42, in execute raise ActionFailedError(str(ex)) aws_lambda_builders.actions.ActionFailedError: {snowflake-connector-python==2.1.2(wheel)} Builder workflow failed Traceback (most recent call last): File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py”, line 39, in execute requirements_path=self.manifest_path File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py”, line 144, in build_dependencies requirements_path, artifacts_dir_path, scratch_dir_path) File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py”, line 213, in build_site_packages raise MissingDependencyError(packages_without_wheels) aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {snowflake-connector-python==2.1.2(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py”, line 248, in run action.execute() File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py”, line 42, in execute raise ActionFailedError(str(ex)) aws_lambda_builders.actions.ActionFailedError: {snowflake-connector-python==2.1.2(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/main.py”, line 137, in main mode=params.get(‘mode’, None)) File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/builder.py”, line 115, in build return workflow.run() File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py”, line 65, in wrapper func(self, *args, **kwargs) File “/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py”, line 257, in run reason=str(ex)) aws_lambda_builders.exceptions.WorkflowFailedError: PythonPipBuilder:ResolveDependencies - {snowflake-connector-python==2.1.2(wheel)} Build inside container returned response {“jsonrpc”: “2.0”, “id”: 1, “error”: {“code”: 400, “message”: “PythonPipBuilder:ResolveDependencies - {snowflake-connector-python==2.1.2(wheel)}”}}

Build Failed Sending Telemetry: {‘metrics’: [{‘commandRun’: {‘awsProfileProvided’: False, ‘debugFlagProvided’: True, ‘region’: ‘’, ‘commandName’: ‘sam build’, ‘duration’: 21152, ‘exitReason’: ‘UserException’, ‘exitCode’: 1, ‘requestId’: ‘bb66e213-3d85-4511-b884-ded8314d29de’, ‘installationId’: ‘9b0b03dd-3c68-43c9-bf8b-4e7849972b58’, ‘sessionId’: ‘b2d3b269-c57d-44e6-ac70-f83bc815c1ef’, ‘executionEnvironment’: ‘CLI’, ‘pyversion’: ‘3.7.5’, ‘samcliVersion’: ‘0.38.0’}}]} HTTPSConnectionPool(host=‘aws-serverless-tools-telemetry.us-west-2.amazonaws.com’, port=443): Read timed out. (read timeout=0.1) Error: PythonPipBuilder:ResolveDependencies - {snowflake-connector-python==2.1.2(wheel)}

---

It seems there is misconfiguration of the wheels dependencies breaking the pip resolver in the AWS lambda docker container.