numba: Generator test segfault on linux py3.11

test_gen9 or test_gen9_objmode in numba.tests.test_generators are either segfaulting or hanging and causing the multiproc testsuite to timeout. Problem only occur on Linux and py3.11. Problem has occurred on x64, ppc64le, and aarch64.

Issue first occurred with the commit 201a2f8, merge of https://github.com/numba/numba/pull/8134.

The issue is hard to reproduce. Only consistent reproduction is in the buildfarm and with the buildfarm created package.

Scripts to reproduce:

$ conda create -y -q -n testenv_f8901e13 -c numba/label/pynext_ci -c numba/label/pynext_dev -c conda-forge python=3.11 numba=0.57.0dev0=np1.23py3.11h3611171_g201a2f8d6_1571 numpy=1.24 pyyaml colorama scipy jinja2 cffi ipython flake8
$ conda activate testenv_f8901e13
$ NUMBA_ENABLE_CUDASIM python -m numba.runtests -m 4 -v numba.tests

conda-forge channel is not necessary. Problem has occurred on py3.11 + np1.23 from conda defaults.

About this issue

Original URL
State: closed
Created a year ago
Comments: 17 (17 by maintainers)

Commits related to this issue

Remove use of `compile_isolated` from generator tests. As title. Use of `compile_isolated` leads to hard to track down life-time bugs such as that in #8873. Essentially the `CodeLibrary` containing t... — committed to stuartarchibald/numba by stuartarchibald a year ago
Remove use of `compile_isolated` from generator tests. As title. Use of `compile_isolated` leads to hard to track down life-time bugs such as that in #8873. Essentially the `CodeLibrary` containing t... — committed to gmarkall/numba by stuartarchibald a year ago

Most upvoted comments

Some backtrace:

#0  __pthread_kill_implementation (no_tid=0, signo=11, threadid=139905809250112)
    at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=11, threadid=139905809250112) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=139905809250112, signo=signo@entry=11) at ./nptl/pthread_kill.c:89
#3  0x00007f3e5c144476 in __GI_raise (sig=11) at ../sysdeps/posix/raise.c:26
#4  <signal handler called>
#5  0x00007f3e588fb080 in ?? ()
#6  0x00007f3e1588f283 in generator_dealloc ()
   from /home/gmarkall/mambaforge/envs/testenv_f8901e13/lib/python3.11/site-packages/numba/_dynfunc.cpython-311-x86_64-linux-gnu.so
#7  0x00005585ddbcbe38 in _Py_Dealloc (op=<optimised out>)
    at /usr/local/src/conda/python-3.11.2/Objects/object.c:2390
#8  Py_DECREF (op=<optimised out>) at /usr/local/src/conda/python-3.11.2/Include/object.h:538
#9  Py_XDECREF (op=<optimised out>) at /usr/local/src/conda/python-3.11.2/Include/object.h:602
#10 _PyFrame_Clear (frame=0x7f3e5c4383a8) at /usr/local/src/conda/python-3.11.2/Python/frame.c:141
#11 0x00005585ddbbaaea in _PyEvalFrameClearAndPop (frame=0x7f3e5c4383a8, 
    tstate=0x5585ddf528b8 <_PyRuntime+166328>)
    at /usr/local/src/conda/python-3.11.2/Python/ceval.c:6407
#12 _PyEval_EvalFrameDefault (tstate=<optimised out>, frame=0x7f3e5c438340, throwflag=<optimised out>)
    at /usr/local/src/conda/python-3.11.2/Python/ceval.c:2439
#13 0x00005585ddc01e5d in _PyEval_EvalFrame (throwflag=0, frame=0x7f3e5c438208, 
    tstate=0x5585ddf528b8 <_PyRuntime+166328>)
    at /usr/local/src/conda/python-3.11.2/Include/internal/pycore_ceval.h:73
#14 _PyEval_Vector (kwnames=<optimised out>, argcount=<optimised out>, args=<optimised out>, 
    locals=0x0, func=0x7f3e150a8fe0, tstate=0x5585ddf528b8 <_PyRuntime+166328>)
    at /usr/local/src/conda/python-3.11.2/Python/ceval.c:6435
#15 _PyFunction_Vectorcall (kwnames=<optimised out>, nargsf=<optimised out>, stack=<optimised out>, 
    func=<function at remote 0x7f3e150a8fe0>) at /usr/local/src/conda/python-3.11.2/Objects/call.c:393
#16 _PyObject_VectorcallTstate (tstate=0x5585ddf528b8 <_PyRuntime+166328>, 
    callable=<function at remote 0x7f3e150a8fe0>, args=<optimised out>, nargsf=<optimised out>, 
    kwnames=<optimised out>) at /usr/local/src/conda/python-3.11.2/Include/internal/pycore_call.h:92
#17 0x00005585ddc01ba5 in method_vectorcall (method=<optimised out>, args=0x7f3e10d50da8, 
    nargsf=<optimised out>, kwnames=0x0) at /usr/local/src/conda/python-3.11.2/Objects/classobject.c:89
#18 0x00005585ddbbed6a in do_call_core (use_tracing=<optimised out>, kwdict={}, 
    callargs=(<TextTestResult(failfast=False, failures=[], errors=[], testsRun=1, skipped=[], expectedFailures=[], unexpectedSuccesses=[], shouldStop=False, buffer=False, tb_locals=False, _stdout_buffer=None, _stderr_buffer=None, _original_stdout=<_io.TextIOWrapper at remote 0x7f3e5bb801e0>, _original_stderr=<_io.TextIOWrapper at remote 0x7f3e5bb802b0>, _mirrorOutput=False, stream=<_WritelnDecorator(stream=<_io.StringIO at remote 0x7f3e58acce50>) at remote 0x7f3e58c69610>, showAll=True, dots=False, descriptions=True, _newline=False) at remote 0x7f3e58ccc350>,), func=<method at remote 0x7f3e58baf040>, 
    tstate=<optimised out>) at /usr/local/src/conda/python-3.11.2/Python/ceval.c:7353
#19 _PyEval_EvalFrameDefault (tstate=<optimised out>, frame=<optimised out>, throwflag=<optimised out>)
    at /usr/local/src/conda/python-3.11.2/Python/ceval.c:5379
#20 0x00005585ddbe21d3 in _PyEval_EvalFrame (throwflag=0, frame=0x7f3e5c438180, 
    tstate=0x5585ddf528b8 <_PyRuntime+166328>)
    at /usr/local/src/conda/python-3.11.2/Include/internal/pycore_ceval.h:73
#21 _PyEval_Vector (kwnames=<optimised out>, argcount=2, args=0x7fff1fd29700, locals=0x0, 
    func=<optimised out>, tstate=0x5585ddf528b8 <_PyRuntime+166328>)
    at /usr/local/src/conda/python-3.11.2/Python/ceval.c:6435
#22 _PyFunction_Vectorcall (func=<optimised out>, stack=0x7fff1fd29700, nargsf=<optimised out>, 
    kwnames=<optimised out>) at /usr/local/src/conda/python-3.11.2/Objects/call.c:393
#23 0x00005585ddbb1cb5 in _PyObject_FastCallDictTstate (tstate=0x5585ddf528b8 <_PyRuntime+166328>, 
    callable=<function at remote 0x7f3e150a91c0>, args=<optimised out>, nargsf=<optimised out>, 
    kwargs=<optimised out>) at /usr/local/src/conda/python-3.11.2/Objects/call.c:141
#24 0x00005585ddbea2d9 in _PyObject_Call_Prepend (tstate=0x5585ddf528b8 <_PyRuntime+166328>, 
    callable=<function at remote 0x7f3e150a91c0>, obj=<optimised out>, args=<optimised out>, 
    kwargs=0x0) at /usr/local/src/conda/python-3.11.2/Objects/call.c:482
#25 0x00005585ddcc16b9 in slot_tp_call (
    self=<TestGenerators(_testMethodName='test_gen9_objmode', _outcome=<_Outcome(expecting_failure=False, result=<TextTestResult(failfast=False, failures=[], errors=[], testsRun=1, skipped=[], expectedFailures=[], unexpectedSuccesses=[], shouldStop=False, buffer=False, tb_locals=False, _stdout_buffer=None, _stderr_buffer=None, _original_stdout=<_io.TextIOWrapper at remote 0x7f3e5bb801e0>, _original_stderr=<_io.TextIOWrapper at remote 0x7f3e5bb802b0>, _mirrorOutput=False, stream=<_WritelnDecorator(stream=<_io.StringIO at remote 0x7f3e58acce50>) at remote 0x7f3e58c69610>, showAll=True, dots=False, descriptions=True, _newline=False) at remote 0x7f3e58ccc350>, result_supports_subtests=True, success=True, expectedFailure=None) at remote 0x7f3e58ccc250>, _testMethodDoc=None, _cleanups=[], _subtest=None, _type_equality_funcs={<type at remote 0x5585ddf189a0>: 'assertDictEqual', <type at remote 0x5585ddf19700>: 'assertListEqual', <type at remote 0x5585ddf126e0>: 'assertTupleEqual', <type at remote 0x5585ddf13a00>: 'assertSetEq...(truncated), 
    args=(<TextTestResult(failfast=False, failures=[], errors=[], testsRun=1, skipped=[], expectedFailures=[], unexpectedSuccesses=[], shouldStop=False, buffer=False, tb_locals=False, _stdout_buffer=None, _stderr_buffer=None, _original_stdout=<_io.TextIOWrapper at remote 0x7f3e5bb801e0>, _original_stderr=<_io.TextIOWrapper at remote 0x7f3e5bb802b0>, _mirrorOutput=False, stream=<_WritelnDecorator(stream=<_io.StringIO at remote 0x7f3e58acce50>) at remote 0x7f3e58c69610>, showAll=True, dots=False, descriptions=True, _newline=False) at remote 0x7f3e58ccc350>,), kwds=0x0)
    at /usr/local/src/conda/python-3.11.2/Objects/typeobject.c:7630
#26 0x00005585ddbae4d4 in _PyObject_MakeTpCall (tstate=0x5585ddf528b8 <_PyRuntime+166328>, 
    callable=<TestGenerators(_testMethodName='test_gen9_objmode', _outcome=<_Outcome(expecting_failure=False, result=<TextTestResult(failfast=False, failures=[], errors=[], testsRun=1, skipped=[], expectedFailures=[], unexpectedSuccesses=[], shouldStop=False, buffer=False, tb_locals=False, _stdout_buffer=None, _stderr_buffer=None, _original_stdout=<_io.TextIOWrapper at remote 0x7f3e5bb801e0>, _original_stderr=<_io.TextIOWrapper at remote 0x7f3e5bb802b0>, _mirrorOutput=False, stream=<_WritelnDecorator(stream=<_io.StringIO at remote 0x7f3e58acce50>) at remote 0x7f3e58c69610>, showAll=True, dots=False, descriptions=True, _newline=False) at remote 0x7f3e58ccc350>, result_supports_subtests=True, success=True, expectedFailure=None) at remote 0x7f3e58ccc250>, _testMethodDoc=None, _cleanups=[], _subtest=None, _type_equality_funcs={<type at remote 0x5585ddf189a0>: 'assertDictEqual', <type at remote 0x5585ddf19700>: 'assertListEqual', <type at remote 0x5585ddf126e0>: 'assertTupleEqual', <type at remote 0x5585ddf13a00>: 'assertSetEq...(truncated), args=<optimised out>, nargs=<optimised out>, keywords=0x0)
    at /usr/local/src/conda/python-3.11.2/Objects/call.c:214

generator_dealloc() (frame #6) probably being a relevant thing of note.

gmarkall on Apr 3, 2023