numcodecs: Illegal instruction (core dumped) with version 0.7.1

Minimal, reproducible code sample, a copy-pastable example if possible

root@248fe153186d:/home/dash# python3
Python 3.8.2 (default, Apr 23 2020, 14:32:57)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numcodecs
Illegal instruction (core dumped)

Problem description

On a slightly older machine I get the above error when importing numcodecs. Does not happen with version 0.6.4 (the last non-yanked version prior to 0.7.1).

Version and installation information

Please provide the following:

Value of numcodecs.__version__ 0.7.1
Version of Python interpreter 3.8.2
Operating system (Linux/Windows/Mac) Linux
How NumCodecs was installed (e.g., “using pip into virtual environment”, or “using conda”) pip3 install numcodecs

Also, if you think it might be relevant, please provide the output from pip list or conda list depending on which was used to install NumCodecs.

Could this be related to #67 in that maybe this machine does not have the AVX instruction set?

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 34 (21 by maintainers)

Most upvoted comments

https://pypi.org/project/numcodecs/0.7.2/ is released. Please let us know if you are still having issues.

joshmoore on Sep 16, 2020

Version 0.7.2 works great for me. Thanks!

dtenenba on Sep 16, 2020

@Czaki, if we set an environment variable in CI, namely DISABLE_NUMCODECS_AVX2, will that be picked up by the wheel build? If not, are there some other steps we need to perform to ensure that environment variable is available for the build?

Do it using CIBW_ENVIRONMENT. https://cibuildwheel.readthedocs.io/en/stable/options/#environment

Normal set of variable will work on MacOs and Windows, but not on linux, because linux wheels are build inside docker container. But CIBW_ENVIRONMENT wil propagate such variable to container.

Czaki on Sep 14, 2020

here is mine for comparision:

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
stepping	: 1
microcode	: 0xb000038
cpu MHz		: 1200.388
cache size	: 35840 KB
physical id	: 0
siblings	: 28
core id		: 0
cpu cores	: 14
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 20
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs taa itlb_multihit
bogomips	: 4800.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

In my there much more flags, but also bugs. Maybe administrator of your machine block avx2 instruction set from security reason. @dtenenba Could you try download proper wheel (cp38) from this build https://github.com/zarr-developers/numcodecs/actions/runs/256439292 and test it?

@jakirkham It looks like AVX2 problem which is available in processor but disabled on virtual machine. I create #253. Did you know if disabling AVX2 is also need for macos?

Czaki on Sep 15, 2020

@jakirkham I think that

pip uninstall numcodecs
pip install -v --no-cache-dir --no-binary numcodecs numcodecs==0.7.1

is simpler (see that I use numcodecs instead of :all:)

And using your approach I suggest

git clone https://github.com/zarr-developers/numcodecs.git
cd numcodecs
git checkout v0.7.1
git submodule update --init --recursive
pip install -e .

Czaki on Sep 14, 2020