arrow: [Python][FlightRPC] Tests segfault on OSX in conda-forge
Describe the bug, including details regarding any error messages, version, and platform.
#14832 switched on the test suite within the conda-builds, which itself synced with https://github.com/conda-forge/arrow-cpp-feedstock/pull/875.
Then as of https://github.com/conda-forge/arrow-cpp-feedstock/pull/975 (synced back in https://github.com/apache/arrow/pull/34652), we needed to add https://github.com/apache/arrow/blob/5e1c112e93092bcbf62c45b51fd6629463332c4d/dev/tasks/conda-recipes/arrow-cpp/meta.yaml#L398-L399
This is a pretty serious issue because potentially the whole flight module is broken/unusable on osx. I’ve raised this point in several discussions around the conda testing here, but never got around to posting an issue.
I’ve retried with arrow 13.0.0 recently and the segfault remains, already in the first test of the test_flight.py
module
test_feather.py .............................ss......................... [ 27%]
.......ss....x........s.....s.. [ 27%]
test_filesystem.py ..... [ 27%]
Fatal Python error: Segmentation fault
Thread 0x0000000109600e00 (most recent call first):
File "$PREFIX/lib/python3.8/site-packages/pyarrow/tests/test_flight.py", line 1000 in test_flight_client_close
File "$PREFIX/lib/python3.8/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
File "$PREFIX/lib/python3.8/site-packages/_pytest/python.py", line 1788 in runtest
File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 262 in <lambda>
File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 341 in from_call
File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 222 in call_and_report
File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 133 in runtestprotocol
File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 324 in _main
File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 270 in wrap_session
File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
File "$PREFIX/lib/python3.8/site-packages/_pytest/config/__init__.py", line 166 in main
File "$PREFIX/lib/python3.8/site-packages/_pytest/config/__init__.py", line 189 in console_main
File "$PREFIX/bin/pytest", line 10 in <module>
/Users/runner/miniforge3/conda-bld/apache-arrow_1690235820790/test_tmp/run_test.sh: line 9: 43620 Segmentation fault: 11 pytest -rfEs -k "not (_not_a_real_test or test_cuda or test_cpp_extension_in_python or test_cython_api or test_visit_strings)"
To fix this, the line quoted above should be removed, and the conda tests still pass.
Component(s)
C++, FlightRPC, Packaging, Python
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 36 (35 by maintainers)
Commits related to this issue
- GH-37621: [Packaging][Conda] Sync conda recipes with feedstocks (#37624) Syncing after the release of 13.0.0 + a couple of migrations (state as of https://github.com/conda-forge/arrow-cpp-feedstock/p... — committed to apache/arrow by h-vetinari 9 months ago
- GH-37621: [Packaging][Conda] Sync conda recipes with feedstocks (#37624) Syncing after the release of 13.0.0 + a couple of migrations (state as of https://github.com/conda-forge/arrow-cpp-feedstock/p... — committed to etseidl/arrow by h-vetinari 9 months ago
- GH-37621: [Packaging][Conda] Sync conda recipes with feedstocks (#37624) Syncing after the release of 13.0.0 + a couple of migrations (state as of https://github.com/conda-forge/arrow-cpp-feedstock/p... — committed to JerAguilon/arrow by h-vetinari 9 months ago
- GH-37621: [Packaging][Conda] Sync conda recipes with feedstocks (#37624) Syncing after the release of 13.0.0 + a couple of migrations (state as of https://github.com/conda-forge/arrow-cpp-feedstock/p... — committed to JerAguilon/arrow by h-vetinari 9 months ago
- GH-37621: [Packaging][Conda] Sync conda recipes with feedstocks (#37624) Syncing after the release of 13.0.0 + a couple of migrations (state as of https://github.com/conda-forge/arrow-cpp-feedstock/p... — committed to loicalleyne/arrow by h-vetinari 9 months ago
- GH-37621: [Packaging][Conda] Sync conda recipes with feedstocks (#37624) Syncing after the release of 13.0.0 + a couple of migrations (state as of https://github.com/conda-forge/arrow-cpp-feedstock/p... — committed to dgreiss/arrow by h-vetinari 9 months ago
A simple C++ app may also reproduce it.
For posterity, to get a debug build of gRPC:
git clean -fdx .
in the grpc-cpp-feedstockenv PATH=... arch -arch x86_64 /bin/bash
(clean your $PATH of any gunk).scripts/run_osx_build.sh
and add--keep-old-work
toconda mambabuild
python3 build-locally.py
ln -s .../grpc-cpp-feedstock/miniforge3/conda-bld/grpc-split_1692042959526/work_moved_libgrpc-1.56.2-h162c7d8_0_osx-64/
back to the original path.../grpc-cpp-feedstock/miniforge3/conda-bld/grpc-split_1692042959526/work
Wait, I’ll create a branch here for 1.56
Thanks, I’ll try that in a bit.
The EXC_BAD_ACCESS is because it’s trying to manipulate an Abseil object, soooooo maybe that’s also involved…