pyzmq: Fails to build on Android, Pixel 3, Termux

Hi!

I tried to install pyzmq (actually pulled as a dependency of jupyterlab) in my Android phone, a Pixel 3, running the Termux Linux environment/terminal emulator.

I tried to install it using:

$ python3 -m pip install --user pyzmq

The failures are to build the C/C++ code under the bundled/ tree.

First, this failed due to the strlcpy() symbol being redefined. Android ships a libc called “bionic” which actually derives from BSD libc, so it does include its own strlcpy().

I downloaded it locally and tried to hack around that error, then I found the next around the tipc code (don’t remember exactly if it was something like tipc_zone or something like that.)

I hacked through that one and the third problem was around pthread_setaffinity_np().

After these three, I got it to build, but then it didn’t seem to work… Running jupyter lab would start spewing errors and they looked related to zmq.

These are the characteristics of the system:

$ uname -a
Linux localhost 4.9.232-gdc3a39a45881-ab6891296 #0 SMP PREEMPT Thu Oct 8 04:02:27 UTC aarch64 Android
$ python3 -V
Python 3.8.5
$ gcc -v
clang version 10.0.1 (https://github.com/termux/termux-packages 2f1431d824f449f09ef50aa365cdd7ed5b8c159d)
Target: aarch-unknown-linux-android
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin

I noticed there are directories under buildutils/include_* (including one for linux-armv). If you’d like me to try to configure my system and send you an appropriate platform.hpp to bundle into pyzmq I’d be happy to help with that.

I actually found a workaround… There’s a package for libzmq in Termux, so I just installed that, with:

$ pkg install libzmq

After that, pip install --user pyzmq worked just fine (I’m assuming it detected the lib was already present) and jupyter lab (which uses pyzmq) worked fine as well.

So perhaps just stopping the install and issuing an instruction to install the binary library would be an appropriate course of action here? Not sure how this can be reliably detected… But breaking with a “Please install xyz” is definitely better than with a puzzling compilation error…

Cheers! Filipe

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 30
Comments: 24 (1 by maintainers)

Links to this issue

https://np.reddit.com/r/termux/comments/kscgfo/jupyter_install_issue_with_clang/gindlzq/

Most upvoted comments

@daviscyd I just tested it on a fresh phone and it worked… Phone was Pixel 2 and didn’t have any of it installed.

Started by installing Termux. Then inside it, I ran:

$ pkg install python

Then:

$ python3 -m pip install --user pyzmq

Which failed with the strlcpy error:

    bundled/zeromq/src/compat.hpp:44:1: error: static declaration of 'strlcpy' follows non-static declaration
    ...
    /data/data/com.termux/files/usr/include/string.h:146:8: note: previous declaration is here
    size_t strlcpy(char* __dst, const char* __src, size_t __n);
    1 error generated.
    error: command '/data/data/com.termux/files/usr/bin/aarch64-linux-android-clang++' failed with exit code 1

Then I installed the library from the binary package:

$ pkg install libzmq

And finally repeated:

$ python3 -m pip install --user pyzmq

Whch worked fine, with:

$ python3 -m pip install --user pyzmq
Collecting pyzmq
  Using cached pyzmq-20.0.0.tar.gz (1.2 MB)
Using legacy 'setup.py install' for pyzmq, since package 'wheel' is not installed.
Installing collected packages: pyzmq
    Running setup.py install for pyzmq ... done
Successfully installed pyzmq-20.0.0

And finally:

$ python3
Python 3.9.1 (default, Dec  9 2020, 13:22:01)
[Clang 9.0.8 (https://android.googlesource.com/toolchain/llvm-project 98c855489 on linux
>>> import zmq
>>> zmq.zmq_version()
'4.3.3'
>>>

filbranden on Jan 8, 2021

@daviscyd No, not that I’m aware of… I simply used the pkg install libzmq and after that python3 -m pip install --user pyzmq worked for me, right away.

I don’t know if there was anything left on my environment, since I ended up downloading the tarball and doing some local pip operations… I don’t think I did left anything, but maybe I did? I guess I could try that on a second phone to see if I can reproduce it… I happen to have one lying around that needs a factory reset, so excellent one to try and see what happens…

In any case, I’m happy to try to help fix this however I can. I don’t know that I understood exactly how the bundled helpers work… But if sending a platform.hpp for this platform would help, I’d be happy to try to provide one (given instructions on how to correctly generate it.)

filbranden on Jan 7, 2021

@filbranden thank you very much, I was in the same situation, and your post saved my day.

anfe67 on Aug 9, 2023

@sacrrie I was having this same problem and compiling with clang instead of arm-linux-androideabi-clang fixed it for me. My exact command was CC=clang python setup.py build and CC=clang pip install pyzmq should also work. You don’t need to compile libzmq from source.

https://github.com/termux/termux-packages/issues/6241#issuecomment-802854734

yubrajbhoi on Mar 19, 2021