Nuitka: macOS: Segfault on module import when using macos

  • Tested with Nuitka 0.6.5 and 0.6.6rc7 and Python 3.6.9, 3.7.4 and 3.7.5
  • Python version managed by miniconda, rest (primarily Nuitka) managed through a pip -m venv virtual environment.
  • macOS 1.14.6 (a coworker just tried macos 1.15.x, same problem)

Minimal example:

Directory mod containing empty __init__.py file. Nuitika run from mod’s parent directory. Afterwards, the mod.so file is moved to different directory where I try to import it.

Same example works fine when running it in a ubuntu:18.04 docker container.

Nuitka run:

(.venv)  % python3 -m nuitka --show-scons --unstripped --debugger --show-modules --module mod
Nuitka:INFO:Included compiled module 'mod'.
Scons command: [...]/.venv/bin/python3 -W ignore [...]/.venv/lib/python3.7/site-packages/nuitka/build/inline_copy/bin/scons.py -f [...]/.venv/lib/python3.7/site-packages/nuitka/build/SingleExe.scons --jobs 8 --warn=no-deprecated --no-site-dir --debug=explain name=mod result_name=mod source_dir=mod.build debug_mode=true python_debug=false unstripped_mode=true module_mode=true full_compat=false experimental= trace_mode=false python_version=3.7 target_arch=x86_64 python_prefix=/usr/local/miniconda3 nuitka_src=[...]/.venv/lib/python3.7/site-packages/nuitka/build nuitka_cache=[...]/Library/Caches/Nuitka module_count=2 show_scons=true abiflags=m
scons: Reading SConscript files ...
Scons: Tools= ['default']
Scons: CC 'gcc' version check:
Scons: CC 'gcc' version check gives '4.2.1' from '4.2.1'
Scons: CC 'cc' version check:
Scons: CC 'cc' version check gives '11.0.0' from 'Apple clang version 11.0.0 (clang-1100.0.33.12)'
Scons: CC 'g++' version check:
Scons: CC 'g++' version check gives '4.2.1' from '4.2.1'
Scons: Initial CC: gcc
Scons: Initial CCVERSION: 4.2.1
Scons: Initial CXX: g++
Scons: Initial CXXVERSION: 4.2.1
Scons: Picked compiler: clang
Scons: Picked name: clang
Scons: Using C11 mode: True
Scons: Compiler used /usr/bin/clang 
Scons: Using resource mode: code
Scons: Told to run compilation on 8 CPUs.
Scons: Launching target: ['mod.so']
scons: done reading SConscript files.
scons: Building targets ...
scons: building `mod.build/module.mod.os' because it doesn't exist
clang -o mod.build/module.mod.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/module.mod.c
scons: building `mod.build/__constants_data.os' because it doesn't exist
clang -o mod.build/__constants_data.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/__constants_data.c
scons: building `mod.build/__constants.os' because it doesn't exist
clang -o mod.build/__constants.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/__constants.c
scons: building `mod.build/__helpers.os' because it doesn't exist
clang -o mod.build/__helpers.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/__helpers.c
scons: building `mod.build/__frozen.os' because it doesn't exist
clang -o mod.build/__frozen.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/__frozen.c
scons: building `mod.build/CompiledCellType.os' because it doesn't exist
clang -o mod.build/CompiledCellType.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledCellType.c
scons: building `mod.build/CompiledFunctionType.os' because it doesn't exist
clang -o mod.build/CompiledFunctionType.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledFunctionType.c
scons: building `mod.build/CompiledMethodType.os' because it doesn't exist
clang -o mod.build/CompiledMethodType.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledMethodType.c
scons: building `mod.build/CompiledGeneratorType.os' because it doesn't exist
clang -o mod.build/CompiledGeneratorType.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledGeneratorType.c
scons: building `mod.build/CompiledCoroutineType.os' because it doesn't exist
clang -o mod.build/CompiledCoroutineType.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledCoroutineType.c
scons: building `mod.build/CompiledAsyncgenType.os' because it doesn't exist
clang -o mod.build/CompiledAsyncgenType.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledAsyncgenType.c
scons: building `mod.build/CompiledFrameType.os' because it doesn't exist
clang -o mod.build/CompiledFrameType.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledFrameType.c
scons: building `mod.build/CompiledCodeHelpers.os' because it doesn't exist
clang -o mod.build/CompiledCodeHelpers.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/CompiledCodeHelpers.c
scons: building `mod.build/InspectPatcher.os' because it doesn't exist
clang -o mod.build/InspectPatcher.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/InspectPatcher.c
scons: building `mod.build/MetaPathBasedLoader.os' because it doesn't exist
clang -o mod.build/MetaPathBasedLoader.os -c -fvisibility=hidden -std=c11 -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -Wunused-but-set-variable -Wall -Werror -Wno-error=strict-aliasing -Wno-strict-aliasing -Wno-error=format -Wno-format -g -O2 -pipe -fPIC -D_XOPEN_SOURCE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D_NUITKA_FROZEN=0 -D_NUITKA_MODULE_COUNT=2 -D_NUITKA_MODULE -I/usr/local/miniconda3/include/python3.7m -I/usr/local/miniconda3/PC -Imod.build -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/include -I[...]/.venv/lib/python3.7/site-packages/nuitka/build/static_src mod.build/MetaPathBasedLoader.c
mscons: building `mod.so' because it doesn't exist
clang -o mod.so -dynamiclib mod.build/module.mod.os mod.build/__constants_data.os mod.build/__constants.os mod.build/__helpers.os mod.build/__frozen.os mod.build/CompiledCellType.os mod.build/CompiledFunctionType.os mod.build/CompiledMethodType.os mod.build/CompiledGeneratorType.os mod.build/CompiledCoroutineType.os mod.build/CompiledAsyncgenType.os mod.build/CompiledFrameType.os mod.build/CompiledCodeHelpers.os mod.build/InspectPatcher.os mod.build/MetaPathBasedLoader.os -L/usr/local/miniconda3/lib -lm -lpython3.7m
scons: done building targets.

Error:

(.venv)  % python -q -X faulthandler 
>>> import mod
Fatal Python error: Segmentation fault

Current thread 0x00000001088b85c0 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1043 in create_module
  File "<frozen importlib._bootstrap>", line 583 in module_from_spec
  File "<frozen importlib._bootstrap>", line 670 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "<stdin>", line 1 in <module>
[1]    21766 segmentation fault  python -q -X faulthandler

Expectation (which works when directly importing the empty module):

(.venv)  % python
Python 3.7.4 (default, Aug 13 2019, 15:17:50) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import mod
>>> 

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 27 (13 by maintainers)

Most upvoted comments

Ok, this is all about macOS by default not lazy linking symbols, which is why I made an exception for macOS to link the shared library, apparently thinking it was needed, and with platform python, you get away with it, as with many linux forms of Python, however not with Anaconda.

Also, it’s wrong, extension modules are not supposed to do that. So this is fixed now on factory, where I finish up a few packaging tweaks before I can release.

Check this out or wait for the pre-release with it http://nuitka.net/doc/factory.html

With that Anaconda 3.6 and 3.7 on macOS are working. Going to try 3.8 soon, but I expect it to be OK as well.