pepper: Error installing via pip

Hello, I am having issues installing pepper via pip on an HPC platform running CentOS 7.7. Note that I do not have admin privileges.

This is the installation command I run (my_root) suryamok@nl001:~/GenomeAssemblyPipeline % python3 -m pip install --user pepper-polish

While it downloads all the requisite files, it is killed at the last step -

  Using cached https://files.pythonhosted.org/packages/91/ed/1044b74956faf3fda73bffaa8223e5d40a549fc360799128fb6ecc9c8428/pepper_polish-0.1.1.tar.gz
Collecting h5py==2.10.0 (from pepper-polish)
  Using cached https://files.pythonhosted.org/packages/3f/c0/abde58b837e066bca19a3f7332d9d0493521d7dd6b48248451a9e3fe2214/h5py-2.10.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting tqdm==4.38.0 (from pepper-polish)
  Using cached https://files.pythonhosted.org/packages/b9/08/8505f192efc72bfafec79655e1d8351d219e2b80b0dec4ae71f50934c17a/tqdm-4.38.0-py2.py3-none-any.whl
Collecting numpy==1.16.6 (from pepper-polish)
  Using cached https://files.pythonhosted.org/packages/17/06/337132f52ae41fca603473f44f4ea100eb030e096da0ea38563a74f63872/numpy-1.16.6-cp37-cp37m-manylinux1_x86_64.whl
Collecting wget==3.2 (from pepper-polish)
  Using cached https://files.pythonhosted.org/packages/47/6a/62e288da7bcda82b935ff0c6cfe542970f04e29c756b0e147251b2fb251f/wget-3.2.zip
Collecting torch==1.4.0 (from pepper-polish)
  Downloading https://files.pythonhosted.org/packages/1a/3b/fa92ece1e58a6a48ec598bab327f39d69808133e5b2fb33002ca754e381e/torch-1.4.0-cp37-cp37m-manylinux1_x86_64.whl (753.4MB)
    99% |████████████████████████████████| 753.4MB 27.7MB/s eta 0:00:01Killed

I would appreciate your help in looking into this.

P.S - I tried installing from source but I think I am missing virtualenv

(my_root) suryamok@nl001:~/GenomeAssemblyPipeline/pepper % make install
test -d venv || virtualenv venv --python=python3 --prompt "(pepper) "
Running virtualenv with interpreter /gstore/apps/Python/3.7.3-GCCcore-6.3.0-bare/bin/python3
/gstore/apps/Python/3.7.3-GCCcore-6.3.0-bare/bin/python3: can't find '__main__' module in '/gstore/apps/Python/3.7.3-GCCcore-6.3.0-bare/lib/python3.7/site-packages/virtualenv-16.6.1-py3.7.egg'
make: *** [Makefile:5: venv/bin/activate] Error 1

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 17 (9 by maintainers)

Most upvoted comments

Working on this - will update as soon as I have results.

kushalsuryamohan on Oct 30, 2020

Hi @kishwarshafin, interesting observation about RACON. I currently have an assembly that’s generated using the following workflow:

Generate long-read draft assembly (Canu)
Polish genome using Illumina data from same animal (PILON) - Typically, I’ve polished 4-5x to improve base call error rates
Hi-C for scaffolding.

However, when annotating the genome, I still find regions of the genome that were not fixed using Illumina data alone and this is not good for gene finding (especially since base call errors could lead to erroneous stop codons inside genes). So, I was planning on using the assembly from (3) as the reference to work through the PEPPER pipeline. Please advice if this is suitable. Much appreciated.

kushalsuryamohan on Oct 21, 2020

@kushalsuryamohan,

Looks like you are missing python3-dev:

Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)

Looks like the node is not configured correctly for python and are missing python3-dev installation. Usually with anaconda they come by default. How do you usually run other programs? Would a docker help you in this case?

Also, if the login node has limited memory then you can’t install this module in there.

kishwarshafin on Oct 19, 2020