snowflake-connector-python: SNOW-634017: MemoryError: Cannot allocate write+execute memory for ffi.callback()

Please answer these questions before submitting your issue. Thanks!

What version of Python are you using?

Python 3.9.13 | packaged by conda-forge

What operating system and processor architecture are you using?

macOS-10.16-arm64-arm-64bit
What are the component versions in the environment (pip freeze)?

cffi==1.15.1 cryptography==37.0.4 snowflake-sqlalchemy==1.3.4 snowflake-connector-python==2.7.9 requests==2.28.1 SQLAlchemy==1.4.39
What did you do? I installed the package with ‘pip install snowflake-connector-python’ and then simply tried to run a query with the created snowflow connector engine. But I got the error

File ~/miniforge3/envs/nftrecsys_tf/lib/python3.9/site-packages/OpenSSL/SSL.py:1128, in Context.set_verify(self, mode, callback)
   1125 if not callable(callback):
   1126     raise TypeError("callback must be callable")
-> 1128 self._verify_helper = _VerifyHelper(callback)
   1129 self._verify_callback = self._verify_helper.callback
   1130 _lib.SSL_CTX_set_verify(self._context, mode, self._verify_callback)

File ~/miniforge3/envs/nftrecsys_tf/lib/python3.9/site-packages/OpenSSL/SSL.py:359, in _VerifyHelper.__init__(self, callback)
    356         else:
    357             return 0
--> 359 self.callback = _ffi.callback(
    360     "int (*)(int, X509_STORE_CTX *)", wrapper
    361 )

MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks

Please help! I had spent a lot of time debugging and tried different workaround. For example, using the suggestion from this post

conda install snowflake-connector-python
conda uninstall --force cffi
pip install cffi

I suspect it’s related to cffi since when I run pip install cffi I still got the error

AssertionError: would build wheel with unsupported tag ('cp39', 'cp39', 'macosx_11_0_arm64')
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cffi
  Running setup.py clean for cffi
Failed to build cffi

Thanks a lot!

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 29 (6 by maintainers)

Most upvoted comments

I believe this is a direct consequence of the problem (with a suggested solution) described in #857

snowflake-connector-python will not run on M1 Macs (Silicon, Arm processor): See #799 and dbt-labs/dbt#3722. That error is due to pyca/pyopenssl#873. pyca/pyopenssl#873 (comment) implies that a fix will not be forthcoming, and that it would be better to swap out the relevant pyopenssl functionality for Python’s built-in SSL library.

@sfc-gh-mkeller What is the proposed solution here then? This is a huge blocker for all M1 Mac users.

EDIT: I am using conda-forge to install my environments. I can confirm that uninstalling the cffi package and reinstalling using pip seems to solve this issue.

conda uninstall --force cffi    
pipe install cffi

Does anyone know why that would fix this issue, and how it can be resolved so that package can be installed through conda? Using pip is not a solution for me.

+13

jordantshaw on Sep 12, 2022

This seems to be a big blocker for M1 Mac users. Any chance to prioritize to fix this?

+13

jpzhangvincent on Sep 8, 2022

I’m using a Macbook M1 with a conda environment for Python 3.10.4. And running conda install "cffi >= 1.15.1=*_3" fixed this issue for me 😃

PD: @aliamirr try this

CxrlosKenobi on Mar 28, 2023

@4sushi I had no issues on my M1 with the YAML I posted above. Try cffi>=1.15.1=*_3 instead of cffi=1.15.1=*_3, and try quoting the * to prevent the shell from interpreting it.

conda create -p test-env python 'cffi>=1.15.1=*_3' snowflake-connector-python

Creating a virtualenv inside a Conda env also seems redundant and unnecessary.

Edit: Does CFFI 1.15.1 v3 support Python 3.8? I am using 3.10 in my project, but I know e.g. Snowpark still requires 3.8.

gwerbin-tive on Dec 21, 2022

@sfc-gh-mkeller while Homebrew Python 3.9 might resolve this, Snowflake’s own Snowpark doesn’t support Python 3.9 (and doesn’t look like it’s going to any time soon), and Homebrew Python 3.8 doesn’t resolve the issue.

badge on Jul 29, 2022

Thank you @sfc-gh-mkeller! I was able to fix this by installing cffi >= 1.15.1=*_3 in my Conda environment:

channels:
  - 'conda-forge'
  - 'defaults'
dependencies:
  - 'python'
  - 'cffi >= 1.15.1=*_3'
  - 'snowflake-connector-python'
  # etc.

Note that Conda Forge has builds for Python 3.8 - 3.10, but apparently not yet for 3.11, for this particular build revision.

gwerbin-tive on Dec 14, 2022

@ironerumi Anthony was talking about snowflake-connector-python while you are talking about snowflake-snowpark-python

sfc-gh-mkeller on Dec 1, 2022

Yep, that is it. We will update the docs. Thank you

sfc-gh-achandrasekaran on Sep 20, 2022

Folks, we have documented a workaround for this issue for Python3.8. Do try it out and let us know.

Thanks for the update, @sfc-gh-achandrasekaran!

Note that the first command in the guide, as it stands, leads to an error:

$ CONDA_SUBDIR=osx-64 conda create -n snowpark python=3.8 -c https://repo.anaconda.com/pkgs/snowflake numpy pandas
usage: conda [-h] [-V] command ...
conda: error: unrecognized arguments: numpy pandas

It should probably be like this:

CONDA_SUBDIR=osx-64 conda create -n snowpark python=3.8 numpy pandas -c https://repo.anaconda.com/pkgs/snowflak

leifericf on Sep 20, 2022

Folks, we have documented a workaround for this issue for Python3.8. Do try it out and let us know.

sfc-gh-achandrasekaran on Sep 16, 2022