snowflake-connector-python: SNOW-395801: MemoryError when connecting on OSX using PyOpenSSL
While trying to establish a connection, I receive a MemoryError
MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks
which appears to be connected to some low level permission error on OSX which is triggered by using PyOpenSSL, at least on M1 mac books
I believe this is caused by https://github.com/pyca/pyopenssl/issues/873
import snowflake.connector
conn = snowflake.connector.connect(
user='whoami',
password="secret_password",
account="account",
)
Python 3.9.6
macOS-11.4-arm64-arm-64bit
Conda list
# conda list
# Name Version Build Channel
abseil-cpp 20210324.2 hbdafb3b_0 conda-forge
anyio 3.2.1 py39h2804cbe_0 conda-forge
appnope 0.1.2 py39h2804cbe_1 conda-forge
argon2-cffi 20.1.0 py39h5161555_2 conda-forge
arrow-cpp 4.0.1 py39h81e7b2a_4_cpu conda-forge
asn1crypto 1.4.0 pyh9f0ad1d_0 conda-forge
async_generator 1.10 py_0 conda-forge
attrs 21.2.0 pyhd8ed1ab_0 conda-forge
aws-c-cal 0.5.11 h4530763_0 conda-forge
aws-c-common 0.6.2 h3422bc3_0 conda-forge
aws-c-event-stream 0.2.7 h9972306_13 conda-forge
aws-c-io 0.10.5 hea86ef8_0 conda-forge
aws-checksums 0.1.11 h487e1a8_7 conda-forge
aws-sdk-cpp 1.8.186 hb86bbe7_3 conda-forge
azure-common 1.1.27 pyhd8ed1ab_0 conda-forge
azure-core 1.14.0 pyhd8ed1ab_0 conda-forge
azure-storage-blob 12.8.1 pyhd8ed1ab_0 conda-forge
babel 2.9.1 pyh44b312d_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
bleach 3.3.1 pyhd8ed1ab_0 conda-forge
blinker 1.4 py_1 conda-forge
boto3 1.18.2 pyhd8ed1ab_0 conda-forge
botocore 1.21.2 pyhd8ed1ab_0 conda-forge
brotlipy 0.7.0 py39h5161555_1001 conda-forge
bzip2 1.0.8 h3422bc3_4 conda-forge
c-ares 1.17.1 h27ca646_1 conda-forge
ca-certificates 2021.5.30 h4653dfc_0 conda-forge
certifi 2020.12.5 py39h2804cbe_1 conda-forge
cffi 1.14.6 py39hda8b47f_0 conda-forge
chardet 3.0.4 py39h0caf4da_1008 conda-forge
charset-normalizer 2.0.0 pyhd8ed1ab_0 conda-forge
cryptography 3.4.7 py39h73257c9_0 conda-forge
debugpy 1.3.0 py39hfb83b0d_0 conda-forge
decorator 5.0.9 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
entrypoints 0.3 pyhd8ed1ab_1003 conda-forge
gflags 2.2.2 hc88da5d_1004 conda-forge
glog 0.5.0 h5c6a83d_0 conda-forge
gmp 6.2.1 h9f76cd9_0 conda-forge
grpc-cpp 1.38.1 h8c8cf55_0 conda-forge
idna 2.10 pyh9f0ad1d_0 conda-forge
importlib-metadata 4.6.1 py39h2804cbe_0 conda-forge
ipykernel 6.0.3 py39h32adebf_0 conda-forge
ipython 7.25.0 py39h32adebf_1 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
isodate 0.6.0 py_1 conda-forge
jedi 0.18.0 py39h2804cbe_2 conda-forge
jinja2 3.0.1 pyhd8ed1ab_0 conda-forge
jmespath 0.10.0 pyh9f0ad1d_0 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 3.2.0 pyhd8ed1ab_3 conda-forge
jupyter_client 6.1.12 pyhd8ed1ab_0 conda-forge
jupyter_core 4.7.1 py39h2804cbe_0 conda-forge
jupyter_server 1.9.0 pyhd8ed1ab_0 conda-forge
jupyterlab 3.0.16 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge
jupyterlab_server 2.6.1 pyhd8ed1ab_0 conda-forge
krb5 1.19.1 hd92b7a7_0 conda-forge
libblas 3.9.0 9_openblas conda-forge
libbrotlicommon 1.0.9 h3422bc3_5 conda-forge
libbrotlidec 1.0.9 h3422bc3_5 conda-forge
libbrotlienc 1.0.9 h3422bc3_5 conda-forge
libcblas 3.9.0 9_openblas conda-forge
libcurl 7.77.0 h8fe1914_0 conda-forge
libcxx 12.0.1 h168391b_0 conda-forge
libedit 3.1.20191231 hc8eb9b7_2 conda-forge
libev 4.33 h642e427_1 conda-forge
libevent 2.1.10 h93437e4_3 conda-forge
libffi 3.3 h9f76cd9_2 conda-forge
libgfortran 5.0.0.dev0 11_0_1_hf114ba7_22 conda-forge
libgfortran5 11.0.1.dev0 hf114ba7_22 conda-forge
liblapack 3.9.0 9_openblas conda-forge
libnghttp2 1.43.0 hf3018f0_0 conda-forge
libopenblas 0.3.15 openmp_hf330de4_1 conda-forge
libprotobuf 3.16.0 hccf11d3_0 conda-forge
libsodium 1.0.18 h27ca646_1 conda-forge
libssh2 1.9.0 hb80f160_6 conda-forge
libthrift 0.14.2 h28a9c34_1 conda-forge
libutf8proc 2.6.1 h27ca646_0 conda-forge
llvm-openmp 12.0.1 hf3c4609_0 conda-forge
lz4-c 1.9.3 h9f76cd9_0 conda-forge
markupsafe 2.0.1 py39h5161555_0 conda-forge
matplotlib-inline 0.1.2 pyhd8ed1ab_2 conda-forge
mistune 0.8.4 py39h5161555_1004 conda-forge
msrest 0.6.21 pyh44b312d_0 conda-forge
nbclassic 0.3.1 pyhd8ed1ab_1 conda-forge
nbclient 0.5.3 pyhd8ed1ab_0 conda-forge
nbconvert 6.1.0 py39h2804cbe_0 conda-forge
nbformat 5.1.3 pyhd8ed1ab_0 conda-forge
ncurses 6.2 h9aa5885_4 conda-forge
nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge
notebook 6.4.0 pyha770c72_0 conda-forge
numpy 1.21.1 py39h1f3b974_0 conda-forge
oauthlib 3.1.1 pyhd8ed1ab_0 conda-forge
openssl 1.1.1k h27ca646_0 conda-forge
orc 1.6.9 h450e963_0 conda-forge
oscrypto 1.2.1 pyhd3deb0d_0 conda-forge
packaging 21.0 pyhd8ed1ab_0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parquet-cpp 1.5.1 2 conda-forge
parso 0.8.2 pyhd8ed1ab_0 conda-forge
pexpect 4.8.0 pyh9f0ad1d_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
prometheus_client 0.11.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.19 pyha770c72_0 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pyarrow 4.0.1 py39h6134bab_4_cpu conda-forge
pycparser 2.20 pyh9f0ad1d_2 conda-forge
pycryptodomex 3.10.1 py39h5161555_0 conda-forge
pygments 2.9.0 pyhd8ed1ab_0 conda-forge
pyjwt 2.1.0 pyhd8ed1ab_0 conda-forge
pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge
pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge
pyrsistent 0.17.3 py39h5161555_2 conda-forge
pysocks 1.7.1 py39h2804cbe_3 conda-forge
python 3.9.6 h54d631c_1_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.9 2_cp39 conda-forge
pytz 2021.1 pyhd8ed1ab_0 conda-forge
pyzmq 22.1.0 py39h02c6a76_0 conda-forge
re2 2021.06.01 hbdafb3b_0 conda-forge
readline 8.1 hedafd6a_0 conda-forge
requests 2.26.0 pyhd8ed1ab_0 conda-forge
requests-oauthlib 1.3.0 pyh9f0ad1d_0 conda-forge
requests-unixsocket 0.2.0 py_0 conda-forge
s3transfer 0.5.0 pyhd8ed1ab_0 conda-forge
send2trash 1.7.1 pyhd8ed1ab_0 conda-forge
setuptools 49.6.0 py39h2804cbe_3 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
snappy 1.1.8 hc88da5d_3 conda-forge
sniffio 1.2.0 py39h2804cbe_1 conda-forge
snowflake-connector-python 2.4.6 py39h1790f5e_0 conda-forge
sqlite 3.36.0 h72a2b83_0 conda-forge
terminado 0.10.1 py39h2804cbe_0 conda-forge
testpath 0.5.0 pyhd8ed1ab_0 conda-forge
tk 8.6.10 hf7e6567_1 conda-forge
tornado 6.1 py39h5161555_1 conda-forge
traitlets 5.0.5 py_0 conda-forge
tzdata 2021a he74cb21_1 conda-forge
urllib3 1.26.6 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 py_1 conda-forge
websocket-client 0.57.0 py39h2804cbe_4 conda-forge
xz 5.2.5 h642e427_1 conda-forge
zeromq 4.3.4 h9f76cd9_0 conda-forge
zipp 3.5.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.11 h31e879b_1009 conda-forge
zstd 1.5.0 h861e0a7_0 conda-forge
-
What did you do? If possible, provide a recipe for reproducing the error. A complete runnable program is good.
-
What did you expect to see?
-
What did you see instead?
Logs
2021-07-20 16:19:58,745 - MainThread connection.py:257 - __init__() - INFO - Snowflake Connector for Python Version: 2.4.6, Python Version: 3.9.6, Platform: macOS-11.4-arm64-arm-64bit
2021-07-20 16:19:58,747 - MainThread connection.py:479 - connect() - DEBUG - connect
2021-07-20 16:19:58,747 - MainThread connection.py:744 - __config() - DEBUG - __config
2021-07-20 16:19:58,748 - MainThread connection.py:868 - __config() - INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
2021-07-20 16:19:58,749 - MainThread connection.py:886 - __config() - INFO - Setting use_openssl_only mode to False
2021-07-20 16:19:58,750 - MainThread converter.py:139 - __init__() - DEBUG - use_numpy: False
2021-07-20 16:19:58,751 - MainThread connection.py:647 - __open_connection() - DEBUG - REST API object was created: nhzosdv-roa32727.snowflakecomputing.com:443
2021-07-20 16:19:58,752 - MainThread auth.py:169 - authenticate() - DEBUG - authenticate
2021-07-20 16:19:58,753 - MainThread auth.py:199 - authenticate() - DEBUG - assertion content: *********
2021-07-20 16:19:58,753 - MainThread auth.py:202 - authenticate() - DEBUG - account=nhzosdv-roa32727, user=fjetter2, database=None, schema=None, warehouse=None, role=None, request_id=bc4a1e83-1bc8-48ba-b267-0636c2585919
2021-07-20 16:19:58,754 - MainThread auth.py:235 - authenticate() - DEBUG - body['data']: {'CLIENT_APP_ID': 'PythonConnector', 'CLIENT_APP_VERSION': '2.4.6', 'SVN_REVISION': None, 'ACCOUNT_NAME': 'nhzosdv-roa32727', 'LOGIN_NAME': 'fjetter2', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'PythonConnector', 'OS': 'Darwin', 'OS_VERSION': 'macOS-11.4-arm64-arm-64bit', 'PYTHON_VERSION': '3.9.6', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'Clang 11.1.0 ', 'OCSP_MODE': 'FAIL_OPEN', 'TRACING': 10, 'LOGIN_TIMEOUT': 120, 'NETWORK_TIMEOUT': None}, 'SESSION_PARAMETERS': {'CLIENT_PREFETCH_THREADS': 4}}
2021-07-20 16:19:58,755 - MainThread retry.py:333 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2021-07-20 16:19:58,756 - MainThread retry.py:333 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2021-07-20 16:19:58,756 - MainThread network.py:1077 - _use_requests_session() - DEBUG - Active requests sessions: 1, idle: 0
2021-07-20 16:19:58,757 - MainThread network.py:752 - _request_exec_wrapper() - DEBUG - remaining request timeout: 120, retry cnt: 1
2021-07-20 16:19:58,758 - MainThread network.py:733 - add_request_guid() - DEBUG - Request guid: 76c45324-b1e2-4a37-8f97-3b2a427e2ef4
2021-07-20 16:19:58,758 - MainThread network.py:911 - _request_exec() - DEBUG - socket timeout: 60
2021-07-20 16:19:58,805 - MainThread connectionpool.py:971 - _new_conn() - DEBUG - Starting new HTTPS connection (1): nhzosdv-roa32727.snowflakecomputing.com:443
2021-07-20 16:19:59,027 - MainThread network.py:1093 - _use_requests_session() - DEBUG - Active requests sessions: 0, idle: 1
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 16 (1 by maintainers)
@sfc-gh-mkeller You’re right, it’s the
cffibinary on conda. I’ve got things working now on my conda env by forcing removal ofcffiand reinstalling it via pip.I had to do this since there were also a whole bunch of things that required
cffiOtherwise, more simply, installing
snowflake-connector-pythonitself with pip works.@sfc-gh-mkeller Really appreciate the quick response. You’re right, it looks like the most recent version of
requestsstill switches on the PyOpenSSL injection if standard library SNI andsslare both missing; and they haven’t dropped support entirely as much as start clearly recommending against it.From https://github.com/pyca/pyopenssl/issues/873#issuecomment-778588689, it sounds like
PyOpenSSLmight eventually fix this, but there hasn’t been much movement since February:Do you have any plans, or see a way forward, to supporting native installations of this package on M1 Macbooks / ARM-based architectures?
@jtcohen6 - Can you share an update on when this might get fixed? So far we can’t get any of these workarounds to work with our new analysts’ M1.