TileDB-SOMA: [Bug][R] Issue with local Conda install

Describe the bug I am trying to install the tiledb-soma R API into a conda environment. When I try to install, the error I get is:

/hpc/reference/gs-conda-envs/tiledb-soma-r/lib/R/library/RcppSpdlog/include/spdlog/sinks/ansicolor_sink-inl.h: In member function 'void spdlog::sinks::ansicolor_sink<ConsoleMutex>::print_range_(const memory_buf_t&, size_t, size_t) [with ConsoleMutex = spdlog::details::console_mutex]':
/hpc/reference/gs-conda-envs/tiledb-soma-r/lib/R/library/RcppSpdlog/include/spdlog/sinks/ansicolor_sink-inl.h:123:11: warning: ignoring return value of 'size_t fwrite(const void*, size_t, size_t, FILE*)' declared with attribute 'warn_unused_result' [-Wunused-result]
  123 |     fwrite(formatted.data() + start, sizeof(char), end - start, target_file_);
      |     ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
x86_64-conda-linux-gnu-c++ -std=gnu++17 -shared -L/hpc/projects/upt/hgcb/managed-envs/tiledb-soma-r/lib/R/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/hpc/projects/upt/hgcb/managed-envs/tiledb-soma-r/lib -Wl,-rpath-link,/hpc/projects/upt/hgcb/managed-envs/tiledb-soma-r/lib -L/hpc/projects/upt/hgcb/managed-envs/tiledb-soma-r/lib -o tiledbsoma.so RcppExports.o nanoarrow.o rinterface.o riterator.o rutilities.o stats.o version.o -ltiledb -L../inst/tiledb/lib -Wl,-rpath,$ORIGIN/../tiledb/lib lib/libtiledbsoma.a -L/hpc/projects/upt/hgcb/managed-envs/tiledb-soma-r/lib/R/lib -lR
/hpc/reference/gs-conda-envs/tiledb-soma-r/bin/../lib/gcc/x86_64-conda-linux-gnu/11.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find lib/libtiledbsoma.a: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [/hpc/projects/upt/hgcb/managed-envs/tiledb-soma-r/lib/R/share/make/shlib.mk:10: tiledbsoma.so] Error 1
ERROR: compilation failed for package ‘tiledbsoma’
* removing ‘/hpc/reference/gs-conda-envs/tiledb-soma-r/lib/R/library/tiledbsoma’

The downloaded source packages are in
	‘/tmp/RtmpyyvokH/downloaded_packages’
Warning message:
In install.packages("tiledbsoma", repos = c("https://tiledb-inc.r-universe.dev",  :
  installation of package ‘tiledbsoma’ had non-zero exit status

To Reproduce

$ mamba --version
mamba 0.22.1
conda 4.12.0
mamba create -n tiledb-soma-r
source activate tiledb-soma-r
mamba install -c conda-forge cmake cxx-compiler
mamba install -c conda-forge arrow tiledb
mamba install -c conda-forge r-arrow r-tiledb
mamba install -c conda-forge r-remotes
install.packages('tiledbsoma', repos = c('https://tiledb-inc.r-universe.dev', 'https://cloud.r-project.org'))

Versions (please complete the following information):

  • TileDB-SOMA version:
  • Language and language version (e.g. Python 3.8, R 4.2.2): R 4.2.3
  • OS (e.g. MacOS, Ubuntu Linux): Centos
  • Note: you can use tiledbsoma.show_package_versions() (Python) or tiledbsoma::show_package_versions() ®

Additional context This is on a HPC system, so don’t have sudo rights or ability to install things outside of a conda env.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (9 by maintainers)

Commits related to this issue

Most upvoted comments

@nguyenv @eddelbuettel it works! Thanks so much. I thought maybe it was conda, but guess it wasn’t. Thanks for being so responsive!

Looks like we’re there now – @nguyenv and I noodled a little more over this and have something.

The root cause was that we had not encountered lib64/ so now we take care of this. For once, Conda is not to blame. 😉

We have a partial solution to this issue.

This problem doesn’t have to do with Conda, but with how different Linux distros install libraries into different lib folders. In Makevars.in, we are linking to lib/libtiledbsoma.a. On Centos (and I believe all Fedora based distros), the static library actually gets installed into lib64/libtiledbsoma.a. We’ll need to update Makevars.in with the patch below to handle these different naming schemas;

diff --git a/apis/r/src/Makevars.in b/apis/r/src/Makevars.in
index 9f0443c..cc7b541 100644
--- a/apis/r/src/Makevars.in
+++ b/apis/r/src/Makevars.in
@@ -4,7 +4,8 @@ CXX_STD = CXX17
 PKG_CPPFLAGS = -I. -I../inst/include/ @tiledb_include@ -DR_BUILD @cxx17_macos@ -Iinclude

 ## We also need the TileDB library
-PKG_LIBS = @cxx17_macos@ @tiledb_libs@ @tiledb_rpath@ lib/libtiledbsoma.a
+TILEDBSOMA_LIB = $(find -iname libtiledbsoma.a)
+PKG_LIBS = @cxx17_macos@ @tiledb_libs@ @tiledb_rpath@ ${TILEDBSOMA_LIB}

 all: $(SHLIB)
         # if we are

With this modification, we now link and build successfully. Unfortunately I am now running into a different problem (on RHEL9 EC2 instance running conda 23.1.0) when the build script runs the CMD R CHECK step.

if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ] && [ -f ../inst/tiledb/lib/libtiledb.dylib ] && [ -f tiledbsoma.so ]; then install_name_tool -change libz.1.dylib @rpath/libz.1.dylib ../inst/tiledb/lib/libtiledb.dylib; install_name_tool -add_rpath @loader_path/../tiledb/lib tiledbsoma.so; fi
installing to /tmp/Rtmp6LRLyr/Rinst1047d66530d5f/00LOCK-tiledbsoma/00new/tiledbsoma/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
Error : MappingBase.Rd:88: unable to load shared object '/tmp/Rtmp6LRLyr/Rinst1047d66530d5f/00LOCK-tiledbsoma/00new/tiledbsoma/libs/tiledbsoma.so':
  /tmp/Rtmp6LRLyr/Rinst1047d66530d5f/00LOCK-tiledbsoma/00new/tiledbsoma/libs/tiledbsoma.so: undefined symbol: _ZN10tiledbsoma15SOMAArrayReader4openESt17basic_string_viewIcSt11char_traitsIcEES4_St3mapINSt7__cxx1112basic_stringIcS3_SaIcEEES9_St4lessIS9_ESaISt4pairIKS9_S9_EEESt6vectorIS9_SaIS9_EES4_S4_St8optionalISC_ImmEE
ERROR: installing Rd objects failed for package 'tiledbsoma'
* removing '/tmp/Rtmp6LRLyr/Rinst1047d66530d5f/tiledbsoma'
      -----------------------------------
ERROR: package installation failed

I do see the symbol present in the static library, and this check step works fine on other distros (specifically Ubuntu and I assume other Debian-based distros), so I am not sure why we are seeing an undefined symbol error here. I will continue to investigate today.

I don’t see a src/Makevars, but I do see a src/Makevars.in

That is normal and expected. As a step in R CMD INSTALL, when a file configure is present (which does not need to be made by autoconf, just named configure and executable) it is run. It then turns any file foo.in into foo.

So please just run ./configure and lets look at the src/Makevars it produces / errors it may encounter.

(We are also looking into creating a feedstock for this to avoid these steps at your end.)

Note that inside the configure script I linked to we do

    if test -d src/libtiledbsoma; then
        ## else with sources build by hand
        cd src
        mkdir -p build
        cmake -B build -S libtiledbsoma -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=. -DOVERRIDE_INSTALL_PREFIX=OFF -DTILEDBSOMA_BUILD_R=ON > cmake_log.txt
        cmake --build build -j 2 >> cmake_log.txt
        cmake --build build --target install-libtiledbsoma >> cmake_log.txt
        rm -rf build
        cd ..
        echo "** static library made"
    fi

and note especially that an ‘are we building for R or not’ variable is there. That always has been, and will continue to be, a problem with using one (shared or static) library for both.

@eddelbuettel Is it possible to build the R package using an existing system libtiledbsoma, analagous to how the r-tiledb recipe works?

--configure-args=--with-tiledb=${CONDA_PREFIX}

@johnkerl Are there plans to add an output for the R client to tiledbsoma-feedstock?