TileDB-SOMA: [Bug] Illegal instruction despite building from source

Describe the bug Calling cellxgene_census.open_soma() and/or python3 -m pytest TileDB-SOMA/apis/python/tests causes an Illegal instruction error. I’ve read through the issues here & see that building from source usually seems to be the resolution, but that hasn’t worked for me (see below). I’ve also tried building from release 1.12 & release 1.11, but got the same error at the same steps in the workflow.

To Reproduce

Creating environment

> mamba env create -n cellxgene_census -f ./environment.yml
> mamba activate cellxgene_census
> uv pip install -r ./requirements.txt
> mamba list
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
_sysroot_linux-64_curr_repodata_hack 3                   h69a702a_16    conda-forge
aiobotocore               2.13.1                   pypi_0    pypi
aiohappyeyeballs          2.3.5                    pypi_0    pypi
aiohttp                   3.10.2                   pypi_0    pypi
aioitertools              0.11.0                   pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
anndata                   0.10.8                   pypi_0    pypi
array-api-compat          1.8                      pypi_0    pypi
async-timeout             4.0.3                    pypi_0    pypi
attrs                     24.2.0                   pypi_0    pypi
binutils                  2.40                 h4852527_7    conda-forge
binutils_impl_linux-64    2.40                 ha1999f0_7    conda-forge
binutils_linux-64         2.40                 hb3c18ed_0    conda-forge
botocore                  1.34.131                 pypi_0    pypi
bzip2                     1.0.8                h4bc722e_7    conda-forge
c-ares                    1.32.3               h4bc722e_0    conda-forge
c-compiler                1.7.0                hd590300_1    conda-forge
ca-certificates           2024.7.4             hbcca054_0    conda-forge
cellxgene-census          1.15.0                   pypi_0    pypi
certifi                   2024.7.4                 pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
clang                     18.1.8          default_h9e3a008_1    conda-forge
clang-18                  18.1.8          default_hf981a13_1    conda-forge
cmake                     3.30.2               hf8c4bd3_0    conda-forge
contourpy                 1.2.1                    pypi_0    pypi
cxx-compiler              1.7.0                h00ab1b0_1    conda-forge
cycler                    0.12.1                   pypi_0    pypi
exceptiongroup            1.2.2                    pypi_0    pypi
fonttools                 4.53.1                   pypi_0    pypi
frozenlist                1.4.1                    pypi_0    pypi
fsspec                    2024.6.1                 pypi_0    pypi
gcc                       12.4.0               h236703b_0    conda-forge
gcc_impl_linux-64         12.4.0               hb2e57f8_0    conda-forge
gcc_linux-64              12.4.0               h6b7512a_0    conda-forge
gxx                       12.4.0               h236703b_0    conda-forge
gxx_impl_linux-64         12.4.0               h613a52c_0    conda-forge
gxx_linux-64              12.4.0               h8489865_0    conda-forge
h5py                      3.11.0                   pypi_0    pypi
icu                       75.1                 he02047a_0    conda-forge
idna                      3.7                      pypi_0    pypi
iniconfig                 2.0.0                    pypi_0    pypi
jmespath                  1.0.1                    pypi_0    pypi
joblib                    1.4.2                    pypi_0    pypi
kernel-headers_linux-64   3.10.0              h4a8ded7_16    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5                    pypi_0    pypi
krb5                      1.21.3               h659f571_0    conda-forge
ld_impl_linux-64          2.40                 hf3520f5_7    conda-forge
legacy-api-wrap           1.4                      pypi_0    pypi
libclang-cpp18.1          18.1.8          default_hf981a13_1    conda-forge
libcurl                   8.9.1                hdb1bdb2_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.2                h59595ed_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     12.4.0             ha4f9413_100    conda-forge
libgcc-ng                 14.1.0               h77fa898_0    conda-forge
libgomp                   14.1.0               h77fa898_0    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libllvm18                 18.1.8               h8b73ec9_2    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libsanitizer              12.4.0               h46f95d5_0    conda-forge
libsqlite                 3.46.0               hde9e2c9_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-devel_linux-64  12.4.0             ha4f9413_100    conda-forge
libstdcxx-ng              14.1.0               hc0a3c3a_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libuv                     1.48.0               hd590300_0    conda-forge
libxml2                   2.12.7               he7c6b58_4    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
llvmlite                  0.43.0                   pypi_0    pypi
make                      4.3                  hd18ef5c_1    conda-forge
matplotlib                3.9.1.post1              pypi_0    pypi
multidict                 6.0.5                    pypi_0    pypi
natsort                   8.4.0                    pypi_0    pypi
ncurses                   6.5                  h59595ed_0    conda-forge
networkx                  3.3                      pypi_0    pypi
numba                     0.60.0                   pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
openssl                   3.3.1                h4bc722e_2    conda-forge
packaging                 24.1                     pypi_0    pypi
pandas                    2.2.2                    pypi_0    pypi
patsy                     0.5.6                    pypi_0    pypi
pillow                    10.4.0                   pypi_0    pypi
pip                       24.2               pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0                    pypi_0    pypi
pyarrow                   17.0.0                   pypi_0    pypi
pyarrow-hotfix            0.6                      pypi_0    pypi
pynndescent               0.5.13                   pypi_0    pypi
pyparsing                 3.1.2                    pypi_0    pypi
pytest                    8.3.2                    pypi_0    pypi
python                    3.10.12         hd12c33a_0_cpython    conda-forge
python-dateutil           2.9.0.post0              pypi_0    pypi
pytz                      2024.1                   pypi_0    pypi
readline                  8.2                  h8228510_1    conda-forge
requests                  2.32.3                   pypi_0    pypi
rhash                     1.4.4                hd590300_0    conda-forge
s3fs                      2024.6.1                 pypi_0    pypi
scanpy                    1.10.2                   pypi_0    pypi
scikit-learn              1.5.1                    pypi_0    pypi
scipy                     1.14.0                   pypi_0    pypi
seaborn                   0.13.2                   pypi_0    pypi
session-info              1.0.0                    pypi_0    pypi
setuptools                72.1.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0                   pypi_0    pypi
somacore                  1.0.11                   pypi_0    pypi
statsmodels               0.14.2                   pypi_0    pypi
stdlib-list               0.10.0                   pypi_0    pypi
sysroot_linux-64          2.17                h4a8ded7_16    conda-forge
threadpoolctl             3.5.0                    pypi_0    pypi
tiledb                    0.29.1                   pypi_0    pypi
tiledbsoma                1.11.4                   pypi_0    pypi
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.0.1                    pypi_0    pypi
tqdm                      4.66.5                   pypi_0    pypi
typeguard                 4.3.0                    pypi_0    pypi
typing-extensions         4.12.2                   pypi_0    pypi
tzdata                    2024.1                   pypi_0    pypi
umap-learn                0.5.6                    pypi_0    pypi
urllib3                   2.2.2                    pypi_0    pypi
uv                        0.2.34                   pypi_0    pypi
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.16.0                   pypi_0    pypi
xz                        5.2.6                h166bdaf_0    conda-forge
yarl                      1.9.4                    pypi_0    pypi
zstd                      1.5.6                ha6fb4c9_0    conda-forge

Building TileDB-SOMA

> git clone https://github.com/single-cell-data/TileDB-SOMA.git # main
> python3 -m uv pip install TileDB-SOMA/apis/python/
Resolved 43 packages in 18.32s
   Built tiledbsoma @ file:///scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
Prepared 1 package in 5m 27s
Uninstalled 3 packages in 649ms
Installed 3 packages in 357ms
 - somacore==1.0.11
 + somacore==1.0.13
 - tiledb==0.29.1
 + tiledb==0.31.1
 - tiledbsoma==1.11.4
 + tiledbsoma==1.12.0rc0.post72.dev619176400 (from file:///scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python)

Running pytest

> python3 -m pytest TileDB-SOMA/apis/python/tests
========================================================================================================== test session starts ===========================================================================================================
platform linux -- Python 3.10.12, pytest-8.3.2, pluggy-1.5.0
rootdir: /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
configfile: pyproject.toml
plugins: typeguard-4.3.0
collected 1766 items                                                                                                                                                                                                                     

TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py Fatal Python error: Illegal instruction

Current thread 0x00007fc1e69fa740 (most recent call first):
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/_collection.py", line 130 in create
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 1007 in _create_or_open_collection
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 505 in from_anndata
  File "/scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py", line 104 in test_import_anndata
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 1627 in runtest
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pytest/__main__.py", line 9 in <module>
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 86 in _run_code
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 196 in _run_module_as_main

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pyarrow.lib, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, tiledb.libtiledb (total: 114)
Illegal instruction

Attempting open_soma()

Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cellxgene_census
>>> soma = cellxgene_census.open_soma()
The "stable" release is currently 2024-07-01. Specify 'census_version="2024-07-01"' in future calls to open_soma() to ensure data consistency.
Illegal instruction

Versions (please complete the following information):

  • TileDB-SOMA version:
>>> tiledbsoma.show_package_versions()
tiledbsoma.__version__              1.12.0rc0.post72.dev619176400
TileDB core version (libtiledbsoma) 2.25.0
python version                      3.10.12.final.0
OS version                          Linux 3.10.0-1160.119.1.el7.x86_64
  • Language and language version: Python 3.10.12
  • OS:
> cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
  • CPU:
> lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
Thread(s) per core:    1
Core(s) per socket:    8
Socket(s):             4
NUMA node(s):          4
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 62
Model name:            Intel(R) Xeon(R) CPU E5-4627 v2 @ 3.30GHz
Stepping:              4
CPU MHz:               3256.091
CPU max MHz:           3600.0000
CPU min MHz:           1200.0000
BogoMIPS:              6599.64
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              16384K
NUMA node0 CPU(s):     0,4,8,12,16,20,24,28
NUMA node1 CPU(s):     1,5,9,13,17,21,25,29
NUMA node2 CPU(s):     2,6,10,14,18,22,26,30
NUMA node3 CPU(s):     3,7,11,15,19,23,27,31
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ssbd rsb_ctxsw ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 39 (25 by maintainers)

Most upvoted comments

build time again shortened to almost instantaneous & did not produce as much output even with the increased verbosity

OK this is somehow not being a build from source then … if it were it would be producing more output, and taking more time

I’ll need to investigate this on a noavx2 system – thank you very much for the experiments @chrish935 …

Here’s the gist from building with a more verbose cmake: https://gist.github.com/chrish935/7c3cd3e48651b54c9fd7460a9ec5adb9.

Turns out that I forgot the -v option in my pip install call.

Note: This is with grabbing a fresh clone of the repo, so we’re doing this without changing the download URL in TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake or replacing the libtiledb.so.2.25 files. Also worth noting that this substantially increased build time compared to when the download URL was changed.

@chrish935 thanks for your time on this.

What you’ve done so far should have worked, so in the words of Sherlock Holmes, when it’s not any of thing things it can be, it has to be one of the things it can’t be.

Something is not what it seems:

  • Our noavx2 tar file isn’t what it seems, or
  • Conflation with another /usr/local/ library is happening in a way I am not seeing, or
  • The instruction which is being illegal at runtime on your system isn’t actually an AVX2 instruction, but some other – gdb can perhaps reveal that

In any event, we at TileDB need to get a system set up similar to this and debug it. I appreciate your generous time involvement but I think we’re at a next step of rolling up the sleeves internally.

@johnkerl Doesn’t it need an updated sha1 for the changed file too?

Still no dice.

> pip uninstall -y tiledbsoma
Found existing installation: tiledbsoma 1.12.0rc0.post72.dev619176400
Uninstalling tiledbsoma-1.12.0rc0.post72.dev619176400:
  Successfully uninstalled tiledbsoma-1.12.0rc0.post72.dev619176400
> code TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake # tiledb-linux-x86_64 -> tiledb-linux-x86_64-noavx2
> cd TileDB-SOMA/apis/python/
> pip install .
Processing /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: attrs>=22.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (24.2.0)
Requirement already satisfied: numba>=0.58.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.60.0)
Requirement already satisfied: numpy<2.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.26.4)
Requirement already satisfied: pandas in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (2.2.2)
Requirement already satisfied: pyarrow-hotfix in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.6)
Requirement already satisfied: scanpy>=1.9.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.10.2)
Requirement already satisfied: scipy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.14.0)
Requirement already satisfied: somacore==1.0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.13)
Requirement already satisfied: tiledb~=0.31.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.31.1)
Requirement already satisfied: typing-extensions in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (4.12.2)
Requirement already satisfied: pyarrow>=9.0.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (17.0.0)
Requirement already satisfied: anndata!=0.10.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.8)
Requirement already satisfied: array-api-compat!=1.5,>1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.8)
Requirement already satisfied: exceptiongroup in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.2)
Requirement already satisfied: h5py>=3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.11.0)
Requirement already satisfied: natsort in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (8.4.0)
Requirement already satisfied: packaging>=20.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (24.1)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from numba>=0.58.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.43.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: joblib in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.2)
Requirement already satisfied: legacy-api-wrap>=1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4)
Requirement already satisfied: matplotlib>=3.6 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.9.1.post1)
Requirement already satisfied: networkx>=2.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.3)
Requirement already satisfied: patsy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: pynndescent>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.13)
Requirement already satisfied: scikit-learn>=0.24 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.5.1)
Requirement already satisfied: seaborn>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.13.2)
Requirement already satisfied: session-info in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.0)
Requirement already satisfied: statsmodels>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.14.2)
Requirement already satisfied: tqdm in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.66.5)
Requirement already satisfied: umap-learn!=0.5.0,>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: contourpy>=1.0.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.5)
Requirement already satisfied: pillow>=8 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (10.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.1.2)
Requirement already satisfied: six>=1.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.16.0)
Requirement already satisfied: threadpoolctl>=3.1.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scikit-learn>=0.24->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.5.0)
Requirement already satisfied: stdlib-list in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from session-info->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.0)
Building wheels for collected packages: tiledbsoma
  Building wheel for tiledbsoma (pyproject.toml) ... done
  Created wheel for tiledbsoma: filename=tiledbsoma-1.12.0rc0.post72.dev619176400-cp310-cp310-linux_x86_64.whl size=1186951 sha256=3d53250beb65d92a13f471ee25d91bb7bec49491f517daf8d7cceaf1fb1d60f9
  Stored in directory: /tmp/pip-ephem-wheel-cache-dmi7fxyo/wheels/17/76/6d/9ce391a1a622a7df9e5e19a14ab6dc1ce279e738d27f3a880d
Successfully built tiledbsoma
Installing collected packages: tiledbsoma
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cellxgene-census 1.15.0 requires tiledbsoma~=1.11.4, but you have tiledbsoma 1.12.0rc0.post72.dev619176400 which is incompatible.
Successfully installed tiledbsoma-1.12.0rc0.post72.dev619176400

Running pytest

> python3 -m pytest ./tests
========================================================================================================== test session starts ===========================================================================================================
platform linux -- Python 3.10.12, pytest-8.3.2, pluggy-1.5.0
rootdir: /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
configfile: pyproject.toml
plugins: typeguard-4.3.0
collected 1766 items                                                                                                                                                                                                                     

tests/test_basic_anndata_io.py Fatal Python error: Illegal instruction

Current thread 0x00007feec2aa7740 (most recent call first):
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/_collection.py", line 130 in create
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 1007 in _create_or_open_collection
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 505 in from_anndata
  File "/scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py", line 104 in test_import_anndata
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 1627 in runtest
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pytest/__main__.py", line 9 in <module>
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 86 in _run_code
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 196 in _run_module_as_main

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pyarrow.lib, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, tiledb.libtiledb (total: 114)
Illegal instruction

Maybe this bit would need a ‘have avx2 or not’ generalization like the one I show above as this will pull the ‘assume AVX2’ blob when we also have one for ‘-noavx2’ in each release build:

https://github.com/single-cell-data/TileDB-SOMA/blob/6b74aa5aa4994777fffa23df1f2b9fcace95491f/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake#L92-L95

OK let me ask around internally for a third pair of 👀 – thanks @chrish935 🙏

Doesn’t appear to be, no.

> ls /usr/local/lib/libtiledb.*
ls: cannot access /usr/local/lib/libtiledb.*: No such file or directory
>>> import tiledbsoma
>>> tiledbsoma.show_package_versions()
tiledbsoma.__version__              1.12.0rc0.post72.dev619176400
TileDB core version (libtiledbsoma) 2.25.0
python version                      3.10.12.final.0
OS version                          Linux 3.10.0-1160.119.1.el7.x86_64

Sure thing.

> pip uninstall -y tiledbsoma
Found existing installation: tiledbsoma 1.11.4
Uninstalling tiledbsoma-1.11.4:
  Successfully uninstalled tiledbsoma-1.11.4
> cd TileDB-SOMA/apis/python/
> pip install .
Processing /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: attrs>=22.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (24.2.0)
Requirement already satisfied: numba>=0.58.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.60.0)
Requirement already satisfied: numpy<2.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.26.4)
Requirement already satisfied: pandas in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (2.2.2)
Requirement already satisfied: pyarrow-hotfix in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.6)
Requirement already satisfied: scanpy>=1.9.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.10.2)
Requirement already satisfied: scipy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.14.0)
Collecting somacore==1.0.13 (from tiledbsoma==1.12.0rc0.post72.dev619176400)
  Using cached somacore-1.0.13-py3-none-any.whl.metadata (1.5 kB)
Collecting tiledb~=0.31.0 (from tiledbsoma==1.12.0rc0.post72.dev619176400)
  Using cached tiledb-0.31.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Requirement already satisfied: typing-extensions in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (4.12.2)
Requirement already satisfied: pyarrow>=9.0.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (17.0.0)
Requirement already satisfied: anndata!=0.10.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.8)
Requirement already satisfied: array-api-compat!=1.5,>1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.8)
Requirement already satisfied: exceptiongroup in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.2)
Requirement already satisfied: h5py>=3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.11.0)
Requirement already satisfied: natsort in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (8.4.0)
Requirement already satisfied: packaging>=20.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (24.1)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from numba>=0.58.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.43.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: joblib in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.2)
Requirement already satisfied: legacy-api-wrap>=1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4)
Requirement already satisfied: matplotlib>=3.6 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.9.1.post1)
Requirement already satisfied: networkx>=2.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.3)
Requirement already satisfied: patsy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: pynndescent>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.13)
Requirement already satisfied: scikit-learn>=0.24 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.5.1)
Requirement already satisfied: seaborn>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.13.2)
Requirement already satisfied: session-info in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.0)
Requirement already satisfied: statsmodels>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.14.2)
Requirement already satisfied: tqdm in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.66.5)
Requirement already satisfied: umap-learn!=0.5.0,>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: contourpy>=1.0.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.5)
Requirement already satisfied: pillow>=8 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (10.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.1.2)
Requirement already satisfied: six>=1.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.16.0)
Requirement already satisfied: threadpoolctl>=3.1.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scikit-learn>=0.24->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.5.0)
Requirement already satisfied: stdlib-list in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from session-info->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.0)
Using cached somacore-1.0.13-py3-none-any.whl (35 kB)
Using cached tiledb-0.31.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.9 MB)
Building wheels for collected packages: tiledbsoma
  Building wheel for tiledbsoma (pyproject.toml) ... done
  Created wheel for tiledbsoma: filename=tiledbsoma-1.12.0rc0.post72.dev619176400-cp310-cp310-linux_x86_64.whl size=1186951 sha256=5ecf4a9c0dbdc7a921346f659f62a96d61359849b9d405732b9ef6eb273bedfd
  Stored in directory: /tmp/pip-ephem-wheel-cache-j4u1wxxl/wheels/17/76/6d/9ce391a1a622a7df9e5e19a14ab6dc1ce279e738d27f3a880d
Successfully built tiledbsoma
Installing collected packages: tiledb, somacore, tiledbsoma
  Attempting uninstall: tiledb
    Found existing installation: tiledb 0.29.1
    Uninstalling tiledb-0.29.1:
      Successfully uninstalled tiledb-0.29.1
  Attempting uninstall: somacore
    Found existing installation: somacore 1.0.11
    Uninstalling somacore-1.0.11:
      Successfully uninstalled somacore-1.0.11
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cellxgene-census 1.15.0 requires tiledbsoma~=1.11.4, but you have tiledbsoma 1.12.0rc0.post72.dev619176400 which is incompatible.
Successfully installed somacore-1.0.13 tiledb-0.31.1 tiledbsoma-1.12.0rc0.post72.dev619176400

Running pytest

> python3 -m pytest ./tests
========================================================================================================== test session starts ===========================================================================================================
platform linux -- Python 3.10.12, pytest-8.3.2, pluggy-1.5.0
rootdir: /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
configfile: pyproject.toml
plugins: typeguard-4.3.0
collected 1766 items                                                                                                                                                                                                                     

tests/test_basic_anndata_io.py Fatal Python error: Illegal instruction

Current thread 0x00007f1ddc40b740 (most recent call first):
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/_collection.py", line 130 in create
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 1007 in _create_or_open_collection
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 505 in from_anndata
  File "/scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py", line 104 in test_import_anndata
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 1627 in runtest
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pytest/__main__.py", line 9 in <module>
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 86 in _run_code
  File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 196 in _run_module_as_main

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pyarrow.lib, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, tiledb.libtiledb (total: 114)
Illegal instruction