spaCy: Illegal hardware instruction in spacy/lang/en/syntax_iterators

One more thing you might try is setting BLIS_ARCH=generic before a fresh pip install. I noticed that the blis build system defaults to linux-x86_64 (even on FreeBSD-{i386,powerpc64}) and this build environment has some specific optimizations like avx2 hard-coded. If you’re running on old hardware, that could be tripping it up. I haven’t looked closely at the other packages, but I wonder if they may be doing the same. With BLIS_ARCH=generic, the basic test passed on my PowerMac G5 under FreeBSD 12.1 and an i386 build succeeded under poudriere (FreeBSD’s containerized port test / package building system).

So I added pip and wheel as dependencies for spacy and srsly. I’ll have to check setup.cfg in all the ports to make sure they’re clean. This allowed the ports spaCy build to download the models, though I have to specify a target to keep it from trying to install to /usr/local/lib/python3.7/site-packages.

Running the script below spewed a lot of what looks like debug output, but I didn’t see any errors. I’m not a spaCy user and don’t know anything about NLP, so if you want to test this further I’ll need feedback from a real user.

#!/bin/sh

for lib in en_core_web_sm de_core_news_sm; do
    if [ ! -e $HOME/spaCy/$lib ]; then
	python3.7 -m spacy download $lib --target=$HOME/spaCy
    fi
done

export PYTHONPATH=/usr/local/lib/python3.7:$HOME/spaCy
python3.7 -v -c \
    "import spacy; spacy_nlp = spacy.load('en_core_web_sm'); spacy_nlp('This is some fine text.')" \
    2>&1 | tee out

Unfortunately, I didn’t see the response back in May.

FWIW, I upgraded the spaCy WIP port at https://github.com/outpaddling/freebsd-ports-wip to 2.3.0.

Instructions for installing from the WIP collection are in the README.

I don’t know how to test spaCy myself - I originally created the port for a researcher I work with - but it builds and installs cleanly.

Good luck,

JB

The error seems to indicate that zsh is crashing, not spaCy. What’s the name of the core file? I would suspect zsh.core.

Illegal instruction generally means it was compiled with optimizations to utilize instructions that are not present on the system running it. So how/where was zsh built/installed and on what processor are you running it?

(grep CPU /var/run/dmesg.boot)

Happy to help when I can.

One other thing you might want to check for is something in your environment, /etc/make.conf, etc. causing builds to use specific optimizations (e.g. AVX) that your CPU does not support. I assume you’re running on the same machine that builds the binaries.

Great to hear it runs fine now, @sr-rolando ! And thanks for all your help, @outpaddling !

Ah, that’s unfortunate. Yea I think none of us on the team have experience with FreeBSD? And it’s not a supported platform. It’s good to know the issue is with Blis, but I’m afraid we don’t really have a solution right now. Sorry! Still, good to hear you can work around it through a Linux machine 😉

I guess I’ll close, yea. Sorry we couldn’t be of more help!

Well, I could boil it down to seeing that it fails in blis, not spaCy. But even recompiling the whole stack down to Python doesn’t resolve the core being dumped.

The current “solution” over here is to move the spaCy application from FreeBSD to some Linux machine. (Not nice but it gets the job done.)

Wondering whether maybe @outpaddling has any idea ?

I’m afraid I really don’t know much more about the details here. Does numpy work as expected?

If you can switch to linux, that might be the simplest way to get spacy up and running.

Thanks for the pointer that the gemm method belongs to blis.

After building blis, srsly, thinc and spaCy from source (in this order) according to their respective README’s and following up with a pip install . in each case, I still get the illegal hardware instruction.

I’ll see to run some test suites of the packages that spaCy depends on. But if that doesn’t reveal anything helpful, I might just have to setup the first Linux machines in the house in order to get the app using spaCy up and running.

(Thanks for your help and pointers so far!)

Hmm, I don’t really know at this point and it’s not one the platforms we support, so it’s hard for us to do any testing.

I would try pip install -U -r requirements.txt and recompiling first to see if that helps (especially updating cython), but otherwise I’m afraid I don’t know. I think you might need to recompile a number of dependencies from source, too, probably thinc, srsly, and blis. Given the gemm error above, see the notes for blis: https://github.com/explosion/cython-blis/#building-blis-for-alternative-architectures

Hmm, to try to narrow things down: does it work if you compile from source as described here? https://github.com/explosion/spaCy/#compile-from-source. After compiling, install with pip install ..