libzmq: Assertion failed: (src/mailbox.cpp:99)
Issue description
I’m from a project called pmacct ( https://github.com/pmacct/pmacct ) which can optionally make use of ZeroMQ for passing messages internally. I got three reports in the last three weeks of users compiling pmacct against ZeroMQ 4.2.5 and getting a failed assertion message back:
shell> nfacctd -f ./nfacctd.detailed.conf Assertion failed: ok (src/mailbox.cpp:99) Aborted (core dumped)
Only an hint: one of the users made some testing and apparently all works well with up to gcc6. But compiling with gcc7 or gcc8 shows the above issue.
Environment
- libzmq version: 4.2.5
- OS: Debian 10, Ubuntu 18.04
Minimal test code / Steps to reproduce the issue
- apt-get install libpcap-dev pkg-config libtool autoconf automake bash
- compile and install ZeroMQ 4.2.5
- git clone https://github.com/pmacct/pmacct.git
- cd pmacct ; ./autogen ; ./configure --enable-zmq; make ; make install
- Edit a nfacctd.conf file, put inside the following line: plugin_pipe_zmq: true
- Start pmacct: nfacctd -f /path/to/nfacctd.conf
- It should pretty much immediately show the failed assertion message
What’s the actual result? (include assertion message & call stack if applicable)
gdb /usr/local/sbin/nfacctd core GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git Copyright © 2018 Free Software Foundation, Inc. < … cut … > Reading symbols from /usr/local/sbin/nfacctd…done. [New LWP 6120] [New LWP 6118] [New LWP 6117] [New LWP 6119] [Thread debugging using libthread_db enabled] Using host libthread_db library “/lib/x86_64-linux-gnu/libthread_db.so.1”. Core was generated by `nfacctd: Core Process [detailed] '. Program terminated with signal SIGABRT, Aborted. #0 __GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:51 51 …/sysdeps/unix/sysv/linux/raise.c: No such file or directory. [Current thread is 1 (Thread 0x7f209000d700 (LWP 6120))] (gdb) where #0 __GI_raise (sig=sig@entry=6) at …/sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007f2092961801 in __GI_abort () at abort.c:79 #2 0x00007f20936b1529 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5 #3 0x00007f20936b60b4 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5 4 0x00007f20936da472 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5 #5 0x00007f20936daf36 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5 #6 0x00007f20936fb5e9 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5 #7 0x00007f20936fb669 in zmq_recv () from /usr/lib/x86_64-linux-gnu/libzmq.so.5 #8 0x000055ec462a574c in p_zmq_recv_str(sock=sock@entry=0x55ec4657fec8 <channels_list+7432>) at zmq_common.c:460 #9 0x000055ec462a58bb in p_zmq_zap_handler (zh=0x55ec4657fec0 <channels_list+7424>) at zmq_common.c:518 #10 0x00007f20936ef554 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.5 #11 0x00007f2092d196db in start_thread (arg=0x7f209000d700) at pthread_create.c:463 #12 0x00007f2092a4288f in clone () at …/sysdeps/unix/sysv/linux/x86_64/clone.S:95 (gdb)
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 2
- Comments: 30 (5 by maintainers)
Links to this issue
Commits related to this issue
- ! fix, zmq_common.c: moved ZAP socket initialization inside the ZAP handler. See: https://github.com/zeromq/libzmq/issues/3313 . — committed to pmacct/pmacct by paololucente 6 years ago
- Fixes for external systems (Jureca & Piz-Daint deployment) * Add deployment scripts * Update packages.yaml and modules.yaml * Update py-sonata-network-reduction package and dependencies * Add late... — committed to BlueBrain/spack by deleted user 4 years ago
- Fixes for external systems (Jureca & Piz-Daint) and BluepyOpt+libzmq (#545) * Add deployment scripts * Update packages.yaml and modules.yaml * Update py-sonata-network-reduction package and depe... — committed to BlueBrain/spack by pramodk 4 years ago
- Fix fork detection on gcc 7 When compiling with gcc 7 and newer, the program produced by AC_CHECK_FUNCS(fork) produces a warning, which results in configure incorrectly disabling fork support. Fix th... — committed to dgloe-hpe/libzmq by dgloe-hpe 3 years ago
- Problem: Fix fork detection on gcc 7 Solution: When compiling with gcc 7 and newer, the program produced by AC_CHECK_FUNCS(fork) produces a warning, which results in configure incorrectly disabling f... — committed to dgloe-hpe/libzmq by dgloe-hpe 3 years ago
For anyone still watching this ticket: zeromq’s configure script is broken for use with gcc v7+ due to gcc introducing a new warning (builtin-declaration-mismatch) which is turned on by default; because zeromq’s configure script turns all warnings into errors for it’s own feature existence tests, the feature tests for fork() and memset() fail. The memset() feature failure is probably just an efficiency issue, but the fork() failure means any program which forks while using zeromq, is liable to fail in various asserts. You can fix the issue by adding “-Wno-builtin-declaration-mismatch” to the configure script lines where “-Werror” is added to the “libzmq_cv_cxx_werror_flag” and doing a distclean.
We were having this issue on multiple clusters and debugging this for quite some time. As @paololucente mentioned:
Everything works fine with GCC v6.1.0 but not with newer GCC 7.3.0 or GCC v8.3.0. So certainly there is an issue.
Thank you very much @paololucente for the hint!
cc: @jorblancoa @matz-e