python-xmlsec: import xmlsec fails on python 3.8/3.9, macOS Big Sur 11.0.1 and M1 chip

xmlsec version: 1.3.9
libxmlsec1 version: 1.2.31
libxml2 version: 2.9.10_2
(venv) tesh$ pkg-config --cflags xmlsec1
-D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_CRYPTO_OPENSSL=1 -I/opt/homebrew/Cellar/libxmlsec1/1.2.31/include/xmlsec1 -I/opt/homebrew/opt/openssl@1.1/include -I/opt/homebrew/opt/openssl@1.1/include/openssl

It installs fine but fails when importing.

In [1]: import xmlsec
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-43dd76db73dd> in <module>
----> 1 import xmlsec

ImportError: dlopen(/Users/tesh/rescale/venv3.9/lib/python3.9/site-packages/xmlsec.cpython-39-darwin.so, 2): Symbol not found: _xmlSecDSigNs
  Referenced from: /Users/tesh/rescale/venv3.9/lib/python3.9/site-packages/xmlsec.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/tesh/rescale/venv3.9/lib/python3.9/site-packages/xmlsec.cpython-39-darwin.so

Python 3.8 and 3.9

$ file $(which python)
/Users/tesh/rescale/venv/bin/python: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
/Users/tesh/rescale/venv/bin/python (for architecture x86_64):	Mach-O 64-bit executable x86_64
/Users/tesh/rescale/venv/bin/python (for architecture arm64):	Mach-O 64-bit executable arm64
$ file $(which python)
/Users/tesh/rescale/venv3.9/bin/python: Mach-O 64-bit executable x86_64
$ otool -L /Users/tesh/rescale/venv3.9/lib/python3.9/site-packages/xmlsec.cpython-39-darwin.so
/Users/tesh/rescale/venv3.9/lib/python3.9/site-packages/xmlsec.cpython-39-darwin.so:
	/usr/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
	/usr/lib/libicucore.A.dylib (compatibility version 1.0.0, current version 66.1.0)

Will really appreciate any help.

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Comments: 37 (4 by maintainers)

Most upvoted comments

I issued with xmlsec.cpython-313-darwin.so, 0x0002): symbol not found in flat namespace '_xmlSecOpenSSLTransformHmacRipemd160GetKlass'

I have Python 3.13 and macos M1. For me helped next steps:

  1. uv remove xmlsec;
  2. uv pip install --no-binary :all: xmlsec

I use uv tool but I think it works for pip and poetry too

Hi there, are there any plans on progressing on this issue anytime soon? I understand the constraints, and would just like to get some feedback so that we can plan for alternatives if needed.

I too have found this very same issue, any fix for this ?

In my further investigation I’ve discovered that when building manually it does not link with homebrew-provided libxmlsec1 libraries:

ld: warning: ignoring file /usr/local/Cellar/libxmlsec1/1.2.33/lib/libxmlsec1-openssl.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file /usr/local/Cellar/libxmlsec1/1.2.33/lib/libxmlsec1.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64

where did you see these ld warnings @vpol ? I presume I’m getting the same problem (otool -L against the built lib is not reporting xmlsec1.dylib) but I’d like to confirm. I’ve tried @cadedaniel 's workaround but it’s not working for me

You can see it if you do pip install -v --no-cache-dir xmlsec. I was able to solve this issue for one of my devs today by installing python, libxml2, libxmlsec1, and pkg-config with arch -x86_64 /usr/local/homebrew/bin/brew, and then adding the following values to ~/.zshrc:

export LDFLAGS="-L/usr/local/opt/libxml2/lib -L/usr/local/opt/libxmlsec1/lib"
export CPPFLAGS="-I/usr/local/opt/libxml2/include -I/usr/local/opt/libxmlsec1/include"

After that, we ran pip install -v --no-cache-dir xmlsec and we were able to use the package in python

We were actually having the opposite problem as https://github.com/mehcode/python-xmlsec/issues/163#issuecomment-1011940875 - we were trying to install xmlsec for x86_64 and it was looking for arm64 packages during the installation. If you’re trying to install it for arm64, find out where the arm64 version of libxmlsec1 is with brew info libxmlsec1, and use that location for the LDFLAGS and CPPFLAGS

Also, we were still getting the ld warning for openssl after adding the LDFLAGS and CPPFLAGS to .zshrc - adding the relevant flags for libxmlsec1-openssl would probably fix that, but we stopped messing with things once it started working so I’m not sure

It installed on my M1 machine. You will need to build the wheel, thus, you will need to install libxmlsec1 and pkg-config. You might need to brew update and reinstall those packages. I have been able to builds the wheels for these versions: 1.3.10 to 1.3.12.

Anyone trying this issue also make sure to only try with Python versions 3.8.10 and greater since that’s when official support was added.

❯ pip install xmlsec==1.3.12 --no-cache-dir
Collecting xmlsec==1.3.12
  Downloading xmlsec-1.3.12.tar.gz (64 kB)
     |████████████████████████████████| 64 kB 3.0 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Requirement already satisfied: lxml>=3.8 in ./.venv/lib/python3.8/site-packages (from xmlsec==1.3.12) (4.6.3)
Building wheels for collected packages: xmlsec
  Building wheel for xmlsec (PEP 517) ... done
  Created wheel for xmlsec: filename=xmlsec-1.3.12-cp38-cp38-macosx_11_0_arm64.whl size=48084 sha256=74d628b8edf4a81bde35e4a0863e2c1d2e83a7f897c0c555c1612e6b25fb1ad4
  Stored in directory: /private/var/folders/j1/d109sd4n55qbk5ty8gzq5jgm0000gn/T/pip-ephem-wheel-cache-rt5nmgzu/wheels/2c/99/a3/98bcb24586c778eb35e23e30af30fbf1c34a0a9ed85e18576a
Successfully built xmlsec
Installing collected packages: xmlsec
Successfully installed xmlsec-1.3.12

❯ pkg-config --cflags xmlsec1
-D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_CRYPTO_OPENSSL=1 -I/opt/homebrew/Cellar/libxmlsec1/1.2.32/include/xmlsec1 -I/opt/homebrew/opt/openssl@1.1/include -I/opt/homebrew/opt/openssl@1.1/include/openssl

❯ python -c "import xmlsec"

❯ file $(which python)
/Users/armenzg/code/sentry/.venv/bin/python: Mach-O 64-bit executable arm64

❯ otool -L .venv/lib/python3.8/site-packages/xmlsec.cpython-38-darwin.so
.venv/lib/python3.8/site-packages/xmlsec.cpython-38-darwin.so:
	/opt/homebrew/opt/libxmlsec1/lib/libxmlsec1-openssl.1.dylib (compatibility version 4.0.0, current version 4.32.0)
	/opt/homebrew/opt/libxmlsec1/lib/libxmlsec1.1.dylib (compatibility version 4.0.0, current version 4.32.0)
	/opt/homebrew/opt/openssl@1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
	/usr/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)
	/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
	/usr/lib/libicucore.A.dylib (compatibility version 1.0.0, current version 66.1.0)

❯ brew info libxmlsec1
libxmlsec1: stable 1.2.32 (bottled)
XML security library
https://www.aleksey.com/xmlsec/
/opt/homebrew/Cellar/libxmlsec1/1.2.32 (220 files, 6.2MB) *
  Poured from bottle on 2021-05-17 at 08:58:46
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/libxmlsec1.rb
License: MIT
==> Dependencies
Build: pkg-config ✔
Required: gnutls ✔, libgcrypt ✘, libxml2 ✔, openssl@1.1 ✔
==> Requirements
Build: Xcode ✘

❯ brew info pkg-config
pkg-config: stable 0.29.2 (bottled)
Manage compile and link flags for libraries
https://freedesktop.org/wiki/Software/pkg-config/
/opt/homebrew/Cellar/pkg-config/0.29.2_3 (11 files, 677.1KB) *
  Poured from bottle on 2021-05-17 at 08:53:19
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/pkg-config.rb
License: GPL-2.0-or-later

Still having this issue on python 3.9 and M1. Any updates?

I get the same issue with python 3.9 on a M1 chip, any updates on this?

@hitesh-raichandani I still can’t reproduce the issue due to missing hardware, but maybe you could attach the build log in the meantime? Simply pip install xmlsec -vvv --force-reinstall --no-cache-dir and drop the full output in the comments.

I’m experiencing this on Python 3.11.11, macOS Sonoma and M1 Max chip

I do as well. symbol not found in flat namespace ‘_xmlSecOpenSSLTransformHmacRipemd160GetKlass’

So what helped was installing --no-binary. Then I got another message: xmlsec.Error: (100, ‘lxml & xmlsec libxml2 library version mismatch’) Specified specif versions: pip install --no-binary=lxml lxml==5.3.0 pip install --no-binary=xmlsec xmlsec==1.3.14 now it works

@subavicky I’m working on a patch, but unfortunately can’t provide ETA

Same issue here. Tried all options you had suggested, nothing worked.

Howdy! Any updates on the progression for this issue? I understand the scenario and would like to hear back regarding workarounds.

Thanks

I’m running into the same issue except I am running python3.7 in the Rosetta2 compatibility mode

I’m experiencing this on Python 3.11.11, macOS Sonoma and M1 Max chip

I do as well. symbol not found in flat namespace ‘_xmlSecOpenSSLTransformHmacRipemd160GetKlass’

Using xmlsec==1.3.12 with python 3.8.12 installed via rosetta-enabled homebrew (python@3.8) worked for me. Thanks @armenzg.