tokenizers: import error Symbol not found: ____chkstk_darwin
I am getting below error when importing tokenizers
.
Details:
- Platform: MacOS 10.13.6
- Installed using
pip
insideconda env
.
ImportError: dlopen(/Users/sumit.jha/.conda/envs/kaggle/lib/python3.7/site-packages/tokenizers/tokenizers.cpython-37m-darwin.so, 2): Symbol not found: ____chkstk_darwin Referenced from: /Users/sumit.jha/.conda/envs/kaggle/lib/python3.7/site-packages/tokenizers/tokenizers.cpython-37m-darwin.so (which was built for Mac OS X 10.15) Expected in: /usr/lib/libSystem.B.dylib in /Users/sumit.jha/.conda/envs/kaggle/lib/python3.7/site-packages/tokenizers/tokenizers.cpython-37m-darwin.so
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 3
- Comments: 24 (9 by maintainers)
same issue here, try pip install tokenizers==0.7.0, then it works
Can’t see any of your screenshots. Anyway, you can install
tokenizers
even if pip says otherwise:See above, it says it is incompatible, and yet it works 🙂
Yes, it is actually compatible, but pip says otherwise because it thinks only
0.8.1.rc1
works withtransformers
. Anyway, you can install0.8.1
which I am sure will work!Awesome! Thank you @sepal, this is extremely helpful.
@Narsil Yes, totally related! I think in our case it is due to our dependencies. If we use only full rust crates, then it seems to work fine for older versions of macOS even when building from the last one. It probably gets funky when we start linking to external libs because I guess the linker does only include what’s necessary for the last version of macOS.
So I guess, this is related to the addition of the
onig
dependency, as it was working right before we added it. Maybelibc
would cause the same thing actually. We should be able to fix it withMACOSX_DEPLOYMENT_TARGET=10.9
added in the env, at least I hope so! (Like here: https://github.com/rust-lang/rustup/issues/24)It only happens in High Sierra, I tried the same thing in a linux system and everything went well
Could it be linked to that ? https://github.com/nodegui/nodegui/issues/391#issuecomment-582788859 Seems building with github actions by defaults make binaries incompatible with older macos versions.