numba: Using TBB with numba fails with pip install, works with apt install

Reporting a bug

I have tried using the latest released version of Numba (most recent is visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
I have included a self contained code sample to reproduce the problem. i.e. it’s possible to run as ‘python bug.py’.

Basically the title. Running the following commands fails:

python3 -m venv env
source env/bin/activate
pip install tbb numba
numba -s | grep TBB

Specifically, this returns:

TBB Threading Layer Available                 : False

The following does work:

sudo apt install libtbb-dev
numba -s | grep TBB

returning

TBB Threading Layer Available                 : True
+-->TBB imported successfully.

Two guesses.

The tbb version on pypi is bricked.
Numba is looking for a specific path that doesn’t exist when installing with pip, but does exist when you install from apt.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 19 (11 by maintainers)

Most upvoted comments

Where is the file located currently? (I see a bunch of libtbb.so files in env/lib). And where does it need to go?

I’m not sure anyone knows for certain. During our issue triage yesterday we discussed this and apparently the location of where to place libraries is a compile-time flag of the python interpreter. This means, the location will most likely be different depending on who compiled your python and thus, where you obtained it from.

Another issue, is that wheels were not designed as a binary packaging format. That is, the tbb package does not ship any Python code, only a compiled shared object / library and it’s “not really” a properly supported use-case. We have asked about this before and you can read more here:

https://github.com/numba/numba/pulls?q=is%3Aopen+is%3Apr+milestone%3A"Numba+0.54+RC"

Incidentally, we actually have a script that unpacks the wheel we generate and removes libtbb.so again and repackages the wheel so that people can (if they are lucky). pip install tbb and have that work.

So, what I would recommend perhaps, is to point the LD_LIBRARY_PATH to the location of the libtbb.so file an try to launch it like that.

Hope that helps!

TL;DR: This works. After you install through pip install --upgrade tbb set your LD_LIBRARY_PATH to the .../venv/lib.

nathzi1505 on May 10, 2023

Where is the file located currently? (I see a bunch of libtbb.so files in env/lib). And where does it need to go?

I’m not sure anyone knows for certain. During our issue triage yesterday we discussed this and apparently the location of where to place libraries is a compile-time flag of the python interpreter. This means, the location will most likely be different depending on who compiled your python and thus, where you obtained it from.

Another issue, is that wheels were not designed as a binary packaging format. That is, the tbb package does not ship any Python code, only a compiled shared object / library and it’s “not really” a properly supported use-case. We have asked about this before and you can read more here:

https://github.com/numba/numba/pulls?q=is%3Aopen+is%3Apr+milestone%3A"Numba+0.54+RC"

Incidentally, we actually have a script that unpacks the wheel we generate and removes libtbb.so again and repackages the wheel so that people can (if they are lucky). pip install tbb and have that work.

So, what I would recommend perhaps, is to point the LD_LIBRARY_PATH to the location of the libtbb.so file an try to launch it like that.

Hope that helps!

esc on Jun 29, 2021