InSilicoSeq: Memory leakage when using abundance file
Hello!
I’ve been using InSilicoSeq to generate mock communities for a project of my own to assess assembly quality (https://github.com/cimendes/LMAS).
To match the distribution of a real community, I’ve computed an abundance file to use with insilicoseq. Unfortunately, when using this option, I have the following issue:
UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.
The iss execution never progresses. I’ve tried running it in a computer note with 250 Gb of available memory and the issue still pressists. Any assistance is very much appreciated.
The command that I’m running:
iss generate --genomes ZymoBIOMICS_genomes.fasta --output LMS --abundance_file Zymos\ mock\ Log\ Samples\ Abundance\ -\ Abundance\ file\ LOG.tsv --cpus 40 -n 95665106 --model miseq
The abundance file passed is available here. The complete genomes are available here
Thank you very much for your assistance!
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 15 (1 by maintainers)
Hi! Did you try the latest release? The new 2.0.0 version has a complete rework of the multiprocessing pipeline which includes a memory leaking fix.
I canceled my job after your post. 😦 I did manage to create a sample using the coverage file without having this memory leakage error, but this option doesn’t allow me to set the total read number so I need to do some math to compensate. If this works the problem is isolated to using an abundance file. Or the combination of using the abundance file and a very high read number. I’ll keep you posted!