cocotb: libgpilog.so: undefined symbol: PyGILState_Release

Greetings. Current master (currently at 5a53f6b) is broken for me (Arch Linux w/ Python 3.8.5).

E.g., running test_vhdl_access returns:

[...]

/usr/bin/ghdl -r  --workdir=sim_build --work=dec_viterbi dec_viterbi_ent --vpi=/home/nicola/.local/lib/python3.8/site-packages/cocotb/libs/libcocotbvpi_ghdl.so  
loading VPI module '/home/nicola/.local/lib/python3.8/site-packages/cocotb/libs/libcocotbvpi_ghdl.so'
/home/nicola/.local/lib/python3.8/site-packages/cocotb/libs/libgpilog.so: undefined symbol: PyGILState_Release
/usr/bin/ghdl:error: cannot load VPI module

[...]

Same for iverilog. git bisect tells me that the responsible commit is 606f2485b082c917195f20b7e5b23ad8dcacd82a by @tpambor (indeed, reverting 606f2485b082c917195f20b7e5b23ad8dcacd82a and ef181522e26c4da6d52ed9443fcbf775155098d5 fixes the issue).

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 16 (16 by maintainers)

Most upvoted comments

You mentioned there were issues runtime loading on Windows due to its use of exported global data. Is that not an issue for us?

Lazy binding of global data does not work but the solution here would be that the embedding library is linked to libpython, therefore no runtime loading, lazy binding, etc. here but we still have control over loading libpython if we load it before the first library dependent on libpython is loaded. If the embedding library is loaded, it will skip loading libpython from disk because these symbols are already present in the process space.

I think it may be best to revert 18c5136 for now, option 2 will take a while to implement.

I’m fine with reverting.

tpambor on Aug 23, 2020

Archlinux:

readelf -r venv/lib/python3.8/site-packages/cocotb/libs/libgpilog.so

Relocation section '.rela.dyn' at offset 0x788 contains 30 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000003d78  000000000008 R_X86_64_RELATIVE                    10e0
000000003d80  000000000008 R_X86_64_RELATIVE                    1090
000000004000  000000000008 R_X86_64_RELATIVE                    4000
000000004028  000000000008 R_X86_64_RELATIVE                    2053
000000004038  000000000008 R_X86_64_RELATIVE                    2059
000000004048  000000000008 R_X86_64_RELATIVE                    205e
000000004058  000000000008 R_X86_64_RELATIVE                    2066
000000004068  000000000008 R_X86_64_RELATIVE                    206c
000000003f50  000100000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_deregisterTMClone + 0
000000003f58  000200000006 R_X86_64_GLOB_DAT 0000000000000000 stdout@GLIBC_2.2.5 + 0
000000003f60  000300000006 R_X86_64_GLOB_DAT 0000000000000000 PyGILState_Release + 0
000000003f68  000400000006 R_X86_64_GLOB_DAT 0000000000000000 vsnprintf@GLIBC_2.2.5 + 0
000000003f70  000500000006 R_X86_64_GLOB_DAT 0000000000000000 _Py_Dealloc + 0
000000003f78  000600000006 R_X86_64_GLOB_DAT 0000000000000000 strlen@GLIBC_2.2.5 + 0
000000003f80  000700000006 R_X86_64_GLOB_DAT 0000000000000000 __stack_chk_fail@GLIBC_2.4 + 0
000000003f88  000800000006 R_X86_64_GLOB_DAT 0000000000000000 PyGILState_Ensure + 0
000000003f90  000900000006 R_X86_64_GLOB_DAT 0000000000000000 fputs@GLIBC_2.2.5 + 0
000000003f98  000a00000006 R_X86_64_GLOB_DAT 0000000000000000 PyErr_Print + 0
000000003fa0  000b00000006 R_X86_64_GLOB_DAT 0000000000000000 fputc@GLIBC_2.2.5 + 0
000000003fa8  000c00000006 R_X86_64_GLOB_DAT 0000000000000000 PyLong_FromLong + 0
000000003fb0  000d00000006 R_X86_64_GLOB_DAT 0000000000000000 fprintf@GLIBC_2.2.5 + 0
000000003fb8  000e00000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
000000003fc0  000f00000006 R_X86_64_GLOB_DAT 0000000000000000 PyObject_CallFunctionO + 0
000000003fc8  001000000006 R_X86_64_GLOB_DAT 0000000000000000 fflush@GLIBC_2.2.5 + 0
000000003fd0  001100000006 R_X86_64_GLOB_DAT 0000000000000000 PyObject_IsTrue + 0
000000003fd8  001200000006 R_X86_64_GLOB_DAT 0000000000000000 PyUnicode_FromString + 0
000000003fe0  001300000006 R_X86_64_GLOB_DAT 0000000000000000 fwrite@GLIBC_2.2.5 + 0
000000003fe8  001400000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_registerTMCloneTa + 0
000000003ff0  001500000006 R_X86_64_GLOB_DAT 0000000000000000 __cxa_finalize@GLIBC_2.2.5 + 0
000000003ff8  001600000006 R_X86_64_GLOB_DAT 0000000000000000 stderr@GLIBC_2.2.5 + 0

Debian:

eadelf -r venv/lib/python3.8/site-packages/cocotb/libs/libgpilog.so 

Relocation section '.rela.dyn' at offset 0x750 contains 14 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000003df0  000000000008 R_X86_64_RELATIVE                    11c0
000000003df8  000000000008 R_X86_64_RELATIVE                    1180
000000004080  000000000008 R_X86_64_RELATIVE                    4080
0000000040a8  000000000008 R_X86_64_RELATIVE                    2058
0000000040b8  000000000008 R_X86_64_RELATIVE                    205e
0000000040c8  000000000008 R_X86_64_RELATIVE                    2063
0000000040d8  000000000008 R_X86_64_RELATIVE                    206b
0000000040e8  000000000008 R_X86_64_RELATIVE                    2071
000000003fd0  000100000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_deregisterTM[...] + 0
000000003fd8  000200000006 R_X86_64_GLOB_DAT 0000000000000000 stdout@GLIBC_2.2.5 + 0
000000003fe0  000a00000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
000000003fe8  001100000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_registerTMCl[...] + 0
000000003ff0  001200000006 R_X86_64_GLOB_DAT 0000000000000000 __cxa_finalize@GLIBC_2.2.5 + 0
000000003ff8  001300000006 R_X86_64_GLOB_DAT 0000000000000000 stderr@GLIBC_2.2.5 + 0

Relocation section '.rela.plt' at offset 0x8a0 contains 13 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000004018  000300000007 R_X86_64_JUMP_SLO 0000000000000000 PyGILState_Release + 0
000000004020  000400000007 R_X86_64_JUMP_SLO 0000000000000000 _Py_Dealloc + 0
000000004028  000500000007 R_X86_64_JUMP_SLO 0000000000000000 strlen@GLIBC_2.2.5 + 0
000000004030  000600000007 R_X86_64_JUMP_SLO 0000000000000000 __stack_chk_fail@GLIBC_2.4 + 0
000000004038  000700000007 R_X86_64_JUMP_SLO 0000000000000000 PyGILState_Ensure + 0
000000004040  000800000007 R_X86_64_JUMP_SLO 0000000000000000 PyErr_Print + 0
000000004048  000900000007 R_X86_64_JUMP_SLO 0000000000000000 PyLong_FromLong + 0
000000004050  000b00000007 R_X86_64_JUMP_SLO 0000000000000000 PyObject_CallFunc[...] + 0
000000004058  000c00000007 R_X86_64_JUMP_SLO 0000000000000000 fflush@GLIBC_2.2.5 + 0
000000004060  000d00000007 R_X86_64_JUMP_SLO 0000000000000000 PyObject_IsTrue + 0
000000004068  000e00000007 R_X86_64_JUMP_SLO 0000000000000000 __vsnprintf_chk@GLIBC_2.3.4 + 0
000000004070  000f00000007 R_X86_64_JUMP_SLO 0000000000000000 PyUnicode_FromString + 0
000000004078  001000000007 R_X86_64_JUMP_SLO 0000000000000000 __fprintf_chk@GLIBC_2.3.4 + 0

On Archlinux non-PLT relocations (R_X86_64_GLOB_DAT) are used for python symbols which cannot be lazy bound whereas on Debian PLT relocations (R_X86_64_JUMP_SLO) are used.

Archlinux passes: -Wl,-z,now:

Marks the object to override the runtime linker’s default mode and require non-lazy runtime binding. This is similar to adding the object to the process by using dlopen(3DL) with the RTLD_NOW mode, or setting the LD_BIND_NOW environment variable in effect.

tpambor on Aug 21, 2020

Library loading looks for me like this:

MODULE= TESTCASE= TOPLEVEL=dec_viterbi_ent TOPLEVEL_LANG=vhdl strace /usr/bin/ghdl -r  --workdir=sim_build --work=dec_viterbi dec_viterbi_ent --vpi=/home/tim/projects/cocotb/venv/lib/python3.8/site-packages/cocotb/libs/libcocotbvpi_ghdl.so 2>&1 | grep -E '^open(at)?(.*)\.so' | grep -v ENOENT
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgnarl-9.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgnat-9.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/tim/projects/cocotb/venv/lib/python3.8/site-packages/cocotb/libs/libcocotbvpi_ghdl.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/tim/projects/cocotb/venv/lib/python3.8/site-packages/cocotb/libs/libgpi.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/tim/projects/cocotb/venv/lib/python3.8/site-packages/cocotb/libs/libgpilog.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/tim/projects/cocotb/venv/lib/python3.8/site-packages/cocotb/libs/libcocotbutils.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/tim/projects/cocotb/venv/lib/python3.8/site-packages/cocotb/libs/libcocotb.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpython3.8.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libexpat.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libffi.so.7", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/tim/projects/cocotb/venv/lib/python3.8/site-packages/cocotb/simulator.cpython-38-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/mmap.cpython-38-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3

$ ldd libgpilog.so
	linux-vdso.so.1 (0x00007ffd4c50c000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0ba0c5b000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f0ba0e3f000)

tpambor on Aug 20, 2020