arrow: [Python] macOS Segfault on Import, Both arm64 and x86_64

Describe the bug, including details regarding any error messages, version, and platform.

The pyarrow packages create a segfault on import when compiled on macOS boxes. This happens on macOS arm64 and x86_64, using (at least) version 12.0.0 and 13.0.0.

Context: I am trying to update the MacPorts apache-arrow ports at https://github.com/macports/macports-ports/pull/19664 and discovered this issue. MacPorts uses the apache-arrow build instructions and approach in python.rst#build-and-test and python_wheel_macos_build.sh.

Behavior:

$ port installed py310-pyarrow
The following ports are currently installed:
  py310-pyarrow @12.0.0_3 (active)

$ PYTHONFAULTHANDLER=1 python3.10 -c 'import pyarrow'
Fatal Python error: Segmentation fault

Current thread 0x00007ff84d123700 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1176 in create_module
  File "<frozen importlib._bootstrap>", line 571 in module_from_spec
  File "<frozen importlib._bootstrap>", line 674 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyarrow/__init__.py", line 65 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<string>", line 1 in <module>
Segmentation fault: 11

Component(s)

Python

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 23 (11 by maintainers)

Most upvoted comments

Whether this is the same issue or a different one, I am seeing a crash on ‘import pyarrow’. But it is a BAD INSTRUCTION crash. Seeing this on both Apple Silicon M1, and Apple Silicon M1 Max. Big Sur and Ventura using the public wheels downloaded via pip. 12.0.1 is the last version that works, 13.0.0, 14.0.0, 14.0.1 and 14.0.2 all crash with the BAD INSTRUCTION. This is not being seen on a M2 Sonoma machine… Are the wheels being built on a M2 machine picking up a higher optimization or processor instruction flag?

Thanks. MacPorts’s apache-arrow already uses these system build flags, so this looks like an issue with jemalloc on macOS.

https://github.com/macports/macports-ports/blob/11afc500de4b609a3ed51465915c8102c8550aac/devel/apache-arrow/Portfile#L156