funannotate: test failure

6 input contigs, 6 larger than 500 bp, N50 is 427,039 bp
Checking duplication of 6 contigs
-----------------------------------------------
scaffold_73 appears duplicated: 100% identity over 100% of the contig. contig length: 15153
scaffold_91 appears duplicated: 100% identity over 100% of the contig. contig length: 8858
scaffold_27 appears duplicated: 100% identity over 100% of the contig. contig length: 427039
-----------------------------------------------
6 input contigs; 6 larger than 500 bp; 3 duplicated; 3 written to file
[Jan 07 06:03 PM]: OS: Ubuntu 22.04, 120 cores, ~ 1057 GB RAM. Python: 3.8.15
[Jan 07 06:03 PM]: Running funanotate v1.8.8
[Jan 07 06:03 PM]: Soft-masking simple repeats with tantan
[Jan 07 06:03 PM]: Repeat soft-masking finished:
Masked genome: /home/xly/test-mask_3e5d9525-c3bd-4e21-a33a-26163f920c78/test.masked.fa
num scaffolds: 2
assembly size: 1,216,048 bp
masked repeats: 50,965 bp (4.19%)
-------------------------------------------------------
-------------------------------------------------------
-------------------------------------------------------
[Jan 07 06:03 PM]: OS: Ubuntu 22.04, 120 cores, ~ 1057 GB RAM. Python: 3.8.15
[Jan 07 06:03 PM]: Running funannotate v1.8.8
Traceback (most recent call last):
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/bin/funannotate", line 8, in <module>
    sys.exit(main())
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
    mod.main(arguments)
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/predict.py", line 472, in main
    for f in os.listdir(os.path.join(LOCALAUGUSTUS, 'species', aug_species)):
FileNotFoundError: [Errno 2] No such file or directory: 'annotate/predict_misc/ab_initio_parameters/augustus/species/saccharomyces'
#########################################################
Running `funannotate clean` unit testing: minimap2 mediated assembly duplications
CMD: funannotate clean -i test.clean.fa -o test.exhaustive.fa --exhaustive
#########################################################
#########################################################
SUCCESS: `funannotate clean` test complete.
#########################################################

#########################################################
Running `funannotate mask` unit testing: RepeatModeler --> RepeatMasker
CMD: funannotate mask -i test.fa -o test.masked.fa --cpus 40
#########################################################
#########################################################
SUCCESS: `funannotate mask` test complete.
#########################################################

#########################################################
Running `funannotate predict` unit testing
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 40 --species Awesome testicus
#########################################################
#########################################################
Traceback (most recent call last):
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/bin/funannotate", line 8, in <module>
    sys.exit(main())
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
    mod.main(arguments)
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 405, in main
    runPredictTest(args)
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 160, in runPredictTest
    assert 1500 <= countGFFgenes(os.path.join(
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 45, in countGFFgenes
    with open(input, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test-predict_3e5d9525-c3bd-4e21-a33a-26163f920c78/annotate/predict_results/Awesome_testicus.gff3'
-----------------------------------------------
6 input contigs, 6 larger than 500 bp, N50 is 427,039 bp
Checking duplication of 6 contigs
-----------------------------------------------
scaffold_73 appears duplicated: 100% identity over 100% of the contig. contig length: 15153
scaffold_91 appears duplicated: 100% identity over 100% of the contig. contig length: 8858
scaffold_27 appears duplicated: 100% identity over 100% of the contig. contig length: 427039
-----------------------------------------------
6 input contigs; 6 larger than 500 bp; 3 duplicated; 3 written to file
[Jan 07 06:03 PM]: OS: Ubuntu 22.04, 120 cores, ~ 1057 GB RAM. Python: 3.8.15
[Jan 07 06:03 PM]: Running funanotate v1.8.8
[Jan 07 06:03 PM]: Soft-masking simple repeats with tantan
[Jan 07 06:03 PM]: Repeat soft-masking finished:
Masked genome: /home/xly/test-mask_5892f6a6-65ca-452d-9ecb-3621e7db9c94/test.masked.fa
num scaffolds: 2
assembly size: 1,216,048 bp
masked repeats: 50,965 bp (4.19%)
-------------------------------------------------------
-------------------------------------------------------
-------------------------------------------------------
[Jan 07 06:03 PM]: OS: Ubuntu 22.04, 120 cores, ~ 1057 GB RAM. Python: 3.8.15
[Jan 07 06:03 PM]: Running funannotate v1.8.8
Traceback (most recent call last):
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/bin/funannotate", line 8, in <module>
    sys.exit(main())
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
    mod.main(arguments)
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/predict.py", line 472, in main
    for f in os.listdir(os.path.join(LOCALAUGUSTUS, 'species', aug_species)):
FileNotFoundError: [Errno 2] No such file or directory: 'annotate/predict_misc/ab_initio_parameters/augustus/species/saccharomyces'
#########################################################
Running `funannotate clean` unit testing: minimap2 mediated assembly duplications
CMD: funannotate clean -i test.clean.fa -o test.exhaustive.fa --exhaustive
#########################################################
#########################################################
SUCCESS: `funannotate clean` test complete.
#########################################################

#########################################################
Running `funannotate mask` unit testing: RepeatModeler --> RepeatMasker
CMD: funannotate mask -i test.fa -o test.masked.fa --cpus 40
#########################################################
#########################################################
SUCCESS: `funannotate mask` test complete.
#########################################################

#########################################################
Running `funannotate predict` unit testing
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 40 --species Awesome testicus
#########################################################
#########################################################
Traceback (most recent call last):
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/bin/funannotate", line 8, in <module>
    sys.exit(main())
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
    mod.main(arguments)
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 405, in main
    runPredictTest(args)
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 160, in runPredictTest
    assert 1500 <= countGFFgenes(os.path.join(
  File "/home/xly/soft/mambaforge-pypy3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 45, in countGFFgenes
    with open(input, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test-predict_5892f6a6-65ca-452d-9ecb-3621e7db9c94/annotate/predict_results/Awesome_testicus.gff3'


-------------------------------------------------------
Checking dependencies for 1.8.8
-------------------------------------------------------
To print all dependencies and versions: funannotate check --show-versions

You are running Python v 3.8.15. Now checking python packages...
All 11 python packages installed


You are running Perl v b'5.032001'. Now checking perl modules...
All 27 Perl modules installed


Checking Environmental Variables...
All 6 environmental variables are set
-------------------------------------------------------
Checking external dependencies...
All 36 external dependencies are installed

(funannotate) xly@wyb-PowerEdge-R920:~$ funannotate check --show-versions
-------------------------------------------------------
Checking dependencies for 1.8.8
-------------------------------------------------------
You are running Python v 3.8.15. Now checking python packages...
biopython: 1.80
goatools: 1.2.3
matplotlib: 3.4.3
natsort: 8.2.0
numpy: 1.24.1
pandas: 1.5.2
psutil: 5.9.4
requests: 2.28.1
scikit-learn: 1.2.0
scipy: 1.10.0
seaborn: 0.12.2
All 11 python packages installed


You are running Perl v b'5.032001'. Now checking perl modules...
Bio::Perl: 1.7.4
Carp: 1.50
Clone: 0.46
DBD::SQLite: 1.72
DBD::mysql: 4.050
DBI: 1.643
DB_File: 1.855
Data::Dumper: 2.183
File::Basename: 2.85
File::Which: 1.24
Getopt::Long: 2.54
Hash::Merge: 0.302
JSON: 4.10
LWP::UserAgent: 6.67
Logger::Simple: 2.0
POSIX: 1.94
Parallel::ForkManager: 2.02
Pod::Usage: 1.69
Scalar::Util::Numeric: 0.40
Storable: 3.15
Text::Soundex: 3.05
Thread::Queue: 3.14
Tie::File: 1.06
URI::Escape: 5.12
YAML: 1.30
threads: 2.25
threads::shared: 1.61
All 27 Perl modules installed


Checking Environmental Variables...
$FUNANNOTATE_DB=/home/xly/soft/mambaforge-pypy3/envs/funannotate/funannotate_db
$PASAHOME=/home/xly/soft/mambaforge-pypy3/envs/funannotate/opt/pasa-2.5.2
$TRINITY_HOME=/home/xly/soft/mambaforge-pypy3/envs/funannotate/opt/trinity-2.8.5
$EVM_HOME=/home/xly/soft/mambaforge-pypy3/envs/funannotate/opt/evidencemodeler-1.1.1
$AUGUSTUS_CONFIG_PATH=/home/xly/soft/mambaforge-pypy3/envs/funannotate/config
$GENEMARK_PATH=/home/xly/soft/mambaforge-pypy3/envs/funannotate/bin
All 6 environmental variables are set
-------------------------------------------------------
Checking external dependencies...
PASA: 2.5.2
CodingQuarry: 2.0
Trinity: 2.8.5
augustus: 3.5.0
bamtools: bamtools 2.5.1
bedtools: bedtools v2.30.0
blat: BLAT v35
diamond: 2.0.15
emapper.py: 2.1.9
ete3: 3.1.2
exonerate: exonerate 2.4.0
fasta: no way to determine
glimmerhmm: 3.0.4
gmap: 2021-08-25
gmes_petap.pl: 4.71_lic
hisat2: 2.2.1
hmmscan: HMMER 3.3.2 (Nov 2020)
hmmsearch: HMMER 3.3.2 (Nov 2020)
java: 17.0.3-internal
kallisto: 0.46.1
mafft: v7.508 (2022/Sep/07)
makeblastdb: makeblastdb 2.2.31+
minimap2: 2.24-r1122
proteinortho: 6.1.7
pslCDnaFilter: no way to determine
salmon: salmon 0.14.1
samtools: samtools 1.16.1
signalp: 5.0b
snap: 2006-07-28
stringtie: 2.2.1
tRNAscan-SE: 2.0.11 (Oct 2022)
tantan: tantan 40
tbl2asn: no way to determine, likely 25.X
tblastn: tblastn 2.2.31+
trimal: trimAl v1.4.rev15 build[2013-12-17]
trimmomatic: 0.39
All 36 external dependencies are installed

I don’t know why stop,and i hope you can reply! thanky very much!

About this issue

Original URL
State: open
Created a year ago
Comments: 27 (10 by maintainers)

Most upvoted comments

I found my error!!! Thanks for the support! @nextgenusfs When I copied the augustus config dir, the parameters dir wasn’t transferred. This resolved it:
rm -r config 
# cp -r $(conda info --base)/envs/funannotate-1.8.13/config .  # didn't even have to do this 
# export AUGUSTUS_CONFIG_PATH=$(pwd)/config # didn't even have to set this 
funannotate test -t all 
I thought augustus config dir has to be writeable, so I copied it (apparently non-fully) and thus introduced a bug.

Hey so it was a problem with the augustus config dir, and it wasn’t the same as ours!

If you own the conda env, you shouldn’t have to do the copy of the config dir. Otherwise, as far as I can tell, users should have to copy the config out and set the AUGUSTUS_CONFIG_PATH manually.

I have the same issue with braker/braker2 for my users, so I run basically the same script as for the funannotate in terms of copying the config dir out and setting the env var for them.

davised on Feb 28, 2023

I found my error!!! Thanks for the support! @nextgenusfs When I copied the augustus config dir, the parameters dir wasn’t transferred. This resolved it:

rm -r config 
# cp -r $(conda info --base)/envs/funannotate-1.8.13/config .  # didn't even have to do this 
# export AUGUSTUS_CONFIG_PATH=$(pwd)/config # didn't even have to set this 
funannotate test -t all

I thought augustus config dir has to be writeable, so I copied it (apparently non-fully) and thus introduced a bug.

jowodo on Feb 17, 2023

Hi all, I run software installs on a HPC and install funannotate using conda. In order to allow multiple users to run the software, I wrote a script that copies the augustus config dir into a user-writeable space.

Therefore, each user will have a copy of the config dir and will be able to write into that dir. Then, the script writes the shell commands to their screen and puts a copy in their home dir so they can source that file and have their augustus config dir set for them.

So that’s the setup for my comments.

I had a user write in that they were having an issue with funannotate and had the same error message as above. I replicated the issue on my end.

I then ran a raw augustus --proteinprofile --species saccharomyes command to see what the error message was, and it was a config dir not found error. My script did not set up the $AUGUSTUS_CONFIG_PATH properly, and the directory was not found.

However, the error message from funannotate does not indicate what the actual augustus error was.

Here’s my question, is there a way to turn on the logging of augustus errors, or forward those errors outside of funannotate presently? If not, can you capture the STDERR from the subprocess call and forward it to us so we can see the issue? I can probably dig through the code and find it to turn it on, but my time is tight at the moment.

Is it possible that there is some other augustus config and/or setting that is causing this error? I don’t think this particular user is having the same problem we were, but if others come looking for help when using a centrally-installed funannotate, it may be difficult for people to figure out why there is an augustus error message.

Thoughts?

davised on Feb 15, 2023