arrow: [Python][FlightRPC] Tests segfault on OSX in conda-forge

Describe the bug, including details regarding any error messages, version, and platform.

#14832 switched on the test suite within the conda-builds, which itself synced with https://github.com/conda-forge/arrow-cpp-feedstock/pull/875.

Then as of https://github.com/conda-forge/arrow-cpp-feedstock/pull/975 (synced back in https://github.com/apache/arrow/pull/34652), we needed to add https://github.com/apache/arrow/blob/5e1c112e93092bcbf62c45b51fd6629463332c4d/dev/tasks/conda-recipes/arrow-cpp/meta.yaml#L398-L399

This is a pretty serious issue because potentially the whole flight module is broken/unusable on osx. I’ve raised this point in several discussions around the conda testing here, but never got around to posting an issue.

I’ve retried with arrow 13.0.0 recently and the segfault remains, already in the first test of the test_flight.py module

test_feather.py .............................ss......................... [ 27%]
.......ss....x........s.....s..                                          [ 27%]
test_filesystem.py .....                                                 [ 27%]
Fatal Python error: Segmentation fault

Thread 0x0000000109600e00 (most recent call first):
  File "$PREFIX/lib/python3.8/site-packages/pyarrow/tests/test_flight.py", line 1000 in test_flight_client_close
  File "$PREFIX/lib/python3.8/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "$PREFIX/lib/python3.8/site-packages/_pytest/python.py", line 1788 in runtest
  File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 341 in from_call
  File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
  File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "$PREFIX/lib/python3.8/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 324 in _main
  File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 270 in wrap_session
  File "$PREFIX/lib/python3.8/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "$PREFIX/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "$PREFIX/lib/python3.8/site-packages/_pytest/config/__init__.py", line 166 in main
  File "$PREFIX/lib/python3.8/site-packages/_pytest/config/__init__.py", line 189 in console_main
  File "$PREFIX/bin/pytest", line 10 in <module>
/Users/runner/miniforge3/conda-bld/apache-arrow_1690235820790/test_tmp/run_test.sh: line 9: 43620 Segmentation fault: 11  pytest -rfEs -k "not (_not_a_real_test or test_cuda or test_cpp_extension_in_python or test_cython_api or test_visit_strings)"

To fix this, the line quoted above should be removed, and the conda tests still pass.

Component(s)

C++, FlightRPC, Packaging, Python

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 36 (35 by maintainers)

Commits related to this issue

Most upvoted comments

A simple C++ app may also reproduce it.

For posterity, to get a debug build of gRPC:

  • git clean -fdx . in the grpc-cpp-feedstock
  • env PATH=... arch -arch x86_64 /bin/bash (clean your $PATH of any gunk)
  • Edit .scripts/run_osx_build.sh and add --keep-old-work to conda mambabuild
  • Edit the recipe to set CMAKE_BUILD_TYPE and also clean CMAKE_BUILD_TYPE out of CMAKE_ARGS (since something in the conda build setup also sets it there)
  • python3 build-locally.py
  • Symlink ln -s .../grpc-cpp-feedstock/miniforge3/conda-bld/grpc-split_1692042959526/work_moved_libgrpc-1.56.2-h162c7d8_0_osx-64/ back to the original path .../grpc-cpp-feedstock/miniforge3/conda-bld/grpc-split_1692042959526/work
  • Install the resulting package, lldb should be able to find debug symbols now

Wait, I’ll create a branch here for 1.56

Thanks, I’ll try that in a bit.

The EXC_BAD_ACCESS is because it’s trying to manipulate an Abseil object, soooooo maybe that’s also involved…