bcbio-nextgen: Samtools error when installing bcbio genome data in bcbio: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory

Version info

• bcbio version (bcbio_nextgen.py --version): 1.2.8
• OS name and version (lsb_release -ds): Ubuntu 20.04.3 LTS

To Reproduce Exact bcbio command you have used:

bcbio_nextgen.py upgrade -u skip --genomes sacCer3 --aligners bwa --aligners bowtie2 --aligners hisat2 --aligners star

This results in the following:

Saccharomyces_cerevisiae.R64-1-1.dna.toplevel.fa.gz  100%[=====================================================================================================================>]   3,61M  3,76MB/s    in 1,0s    

2021-11-14 20:56:19 (3,76 MB/s) - written to stdout [3786555]

Sorted contigs saved to "seq/sacCer3.fa" ... 
samtools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/home/user/bcbio/anaconda/bin/bcbio_nextgen.py", line 228, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/home/user/bcbio/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 107, in upgrade_bcbio
    upgrade_bcbio_data(args, REMOTES)
  File "/home/user/bcbio/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 359, in upgrade_bcbio_data
    args.cores, ["ggd", "s3", "raw"])
  File "/home/user/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
    _prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
  File "/home/user/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
    retrieve_fn(env, manager, gid, idx)
  File "/home/user/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
    ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
  File "/home/user/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
    recipe["recipe"]["full"]["recipe_type"], system_install)
  File "/home/user/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
    subprocess.check_output(["bash", run_file])
  File "/home/user/bcbio/anaconda/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/home/user/bcbio/anaconda/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/home/user/bcbio/genomes/Scerevisiae/sacCer3/txtmp/ggd-run.sh']' returned non-zero exit status 127.

This suggests an older version of samtools is being used at some point. There are multiple versions of samtools installed in bcbio:

find bcbio -name samtools
bcbio/anaconda/pkgs/manta-1.6.0-h9ee0642_1/share/manta-1.6.0-1/libexec/samtools
bcbio/anaconda/pkgs/pysam-0.15.3-py27hbcae180_3/lib/python2.7/site-packages/pysam/include/samtools
bcbio/anaconda/pkgs/samtools-1.10-h2e538c0_3/bin/samtools
bcbio/anaconda/pkgs/pysam-0.15.4-py36h873a209_1/lib/python3.6/site-packages/pysam/include/samtools
bcbio/anaconda/pkgs/samtools-1.7-1/bin/samtools
bcbio/anaconda/pkgs/pysam-0.17.0-py37h45aed0b_0/lib/python3.7/site-packages/pysam/include/samtools
bcbio/anaconda/pkgs/multiqc-1.11-pyhdfd78af_0/site-packages/multiqc/modules/samtools
bcbio/anaconda/pkgs/samtools-0.1.19-h270b39a_9/bin/samtools
bcbio/anaconda/pkgs/samtools-1.13-h8c37831_0/bin/samtools
bcbio/anaconda/pkgs/samtools-1.12-h9aed4be_1/bin/samtools
bcbio/anaconda/pkgs/strelka-2.9.10-h9ee0642_1/share/strelka-2.9.10-1/libexec/samtools
bcbio/anaconda/pkgs/bioconductor-rsamtools-1.34.0-r351hf484d3e_0/lib/R/library/Rsamtools/include/samtools
bcbio/anaconda/lib/python3.7/site-packages/pysam/include/samtools
bcbio/anaconda/lib/python3.7/site-packages/multiqc/modules/samtools
bcbio/anaconda/bin/samtools
bcbio/anaconda/envs/htslib1.10/bin/samtools
bcbio/anaconda/envs/python2/share/manta-1.6.0-1/libexec/samtools
bcbio/anaconda/envs/python2/share/strelka-2.9.10-1/libexec/samtools
bcbio/anaconda/envs/python2/lib/python2.7/site-packages/pysam/include/samtools
bcbio/anaconda/envs/python2/bin/samtools
bcbio/anaconda/envs/bwakit/bin/samtools
bcbio/anaconda/envs/python3.6/lib/python3.6/site-packages/pysam/include/samtools
bcbio/anaconda/envs/python3.6/bin/samtools
bcbio/anaconda/envs/htslib1.12_py3.9/bin/samtools
bcbio/anaconda/envs/r35/lib/R/library/Rsamtools/include/samtools
bcbio/anaconda/envs/r35/bin/samtools
bcbio/anaconda/envs/samtools0/bin/samtools
bcbio/tools/bin/samtools

My PATH is configured like this:

export PATH=/home/user/bcbio/anaconda/bin:/home/user/bcbio/tools/bin:$PATH

And the main version of Samtools on my system, given the above, seems to be v1.13:

$ which samtools
/home/user/bcbio/anaconda/bin/samtools
$ samtools --version
samtools 1.13
Using htslib 1.13
Copyright (C) 2021 Genome Research Ltd.

There also seems to be an older version of Samtools getting symlinked in my bcbio tools directory. My guess is that this is probably being picked up somehow when installing the genome data.

$ /home/user/bcbio/tools/bin/samtools
/home/user/bcbio/tools/bin/samtools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory
$ ls -l /home/user/bcbio/tools/bin/samtools
lrwxrwxrwx 1 user user 42 nov 14 20:04 /home/user/bcbio/tools/bin/samtools -> ../../anaconda/envs/python3.6/bin/samtools

Removing this symlink from the tools directory got things working for me and I managed to install the genome data using the command mentioned above.