bcbio-nextgen: Ensemble error when adding vardict

I tried to run a pipeline on germline data using gatk strelka2 and vardict.

Running only with gatk AND strelka2 finished fine.

Adding vardict gave me the following error in many samples:

[E::bcf_write] Broken VCF record, the number of columns at chrY:26641147 does not match the number of samples (0 vs 1)
' returned non-zero exit status 255.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/RED/TOOLS/bin/bcbio_nextgen.py", line 245, in <module>
    main(**kwargs)
  File "/RED/TOOLS/bin/bcbio_nextgen.py", line 46, in main
    run_main(**kwargs)
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 50, in run_main
    fc_dir, run_info_yaml)
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 91, in _run_toplevel
    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 154, in variant2pipeline
    samples = genotype.parallel_variantcall_region(samples, run_parallel)
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/bcbio/variation/genotype.py", line 208, in parallel_variantcall_region
    "vrn_file", ["region", "sam_ref", "config"]))
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/split.py", line 35, in grouped_parallel_split_combine
    final_output = parallel_fn(parallel_name, split_args)
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"], batch_size=1, backend="multiprocessing")(joblib.delayed(fn)(*x) for x in items):
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 1016, in __call__
    self.retrieve()
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 908, in retrieve
    self._output.extend(job.get(timeout=self.timeout))
  File "/RED/RESOURCES/bcbio/anaconda/lib/python3.6/multiprocessing/pool.py", line 670, in get
    raise self._value
subprocess.CalledProcessError: Command 'set -o pipefail; unset R_HOME && unset R_LIBS && export PATH=/RED/RESOURCES/bcbio/anaconda/bin:"$PATH" && unset JAVA_HOME &&export VAR_DICT_OPTS='-Xms750m -Xmx3500m -XX:+UseSerialGC -Djava.io.tmpdir=/mnt/36642bae-9ec9-4100-88a2-ac173a20ea16/WORKDIR/BRAZILIANS/1781F/work/bcbiotx/tmph4gpqeo4' && vardict-java -G /RED/RESOURCES/bcbio/genomes/Hsapiens/hg38/seq/hg38.fa -N 1781F -b /mnt/36642bae-9ec9-4100-88a2-ac173a20ea16/WORKDIR/BRAZILIANS/1781F/work/align/1781F/1781F-sort.bam -c 1 -S 2 -E 3 -g 4 --nosv --deldupvar -Q 10 -F 0x700 /mnt/36642bae-9ec9-4100-88a2-ac173a20ea16/WORKDIR/BRAZILIANS/1781F/work/vardict/chrY/1781F-chrY_16481226_56685608-unmerged-regions-regionlimit.bed -f 0.1 | /RED/RESOURCES/bcbio/anaconda/bin/teststrandbias.R | /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl -A -N 1781F -E -f 0.1 | grep -v ^##contig | bcftools annotate -h /mnt/36642bae-9ec9-4100-88a2-ac173a20ea16/WORKDIR/BRAZILIANS/1781F/work/bcbiotx/tmph4gpqeo4/1781F-chrY_16481226_56685608-contig_header.txt | bcftools filter -i 'QUAL >= 0'  | bcftools filter --soft-filter 'LowFreqBias' --mode '+' -e  'FORMAT/AF[0:0] < 0.02 && FORMAT/VD[0] < 30 && INFO/SBF < 0.1 && INFO/NM >= 2.0' |  awk -F$'\t' -v OFS='\t' '{if ($0 !~ /^#/) gsub(/[KMRYSWBVHDXkmryswbvhdx]/, "N", $4) } {print}' | awk -F$'\t' -v OFS='\t' '{if ($0 !~ /^#/) gsub(/[KMRYSWBVHDXkmryswbvhdx]/, "N", $5) } {print}' |  awk -F$'\t' -v OFS='\t' '$1!~/^#/ && $4 == $5 {next} {print}' | /RED/RESOURCES/bcbio/anaconda/bin/vcfstreamsort | bgzip -c > /mnt/36642bae-9ec9-4100-88a2-ac173a20ea16/WORKDIR/BRAZILIANS/1781F/work/bcbiotx/tmph4gpqeo4/1781F-chrY_16481226_56685608.vcf.gz
Use of uninitialized value $start in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 138, <> line 32.
Use of uninitialized value $end in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 138, <> line 32.
Use of uninitialized value $ref in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 138, <> line 32.
Use of uninitialized value $alt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 138, <> line 32.
Use of uninitialized value $rrev in addition (+) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 141, <> line 32.
Use of uninitialized value $rfwd in addition (+) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 141, <> line 32.
Use of uninitialized value $oddratio in string eq at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 142, <> line 32.
Use of uninitialized value $oddratio in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 144, <> line 32.
Use of uninitialized value $oddratio in numeric gt (>) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 144, <> line 32.
Use of uninitialized value $dp in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 148, <> line 32.
Use of uninitialized value $hiaf in multiplication (*) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 149, <> line 32.
Use of uninitialized value $hicnt in multiplication (*) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 149, <> line 32.
Use of uninitialized value $hicnt in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 151, <> line 32.
Use of uninitialized value $hiaf in multiplication (*) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 152, <> line 32.
Use of uninitialized value $hicnt in multiplication (*) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 152, <> line 32.
Use of uninitialized value $af in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 154, <> line 32.
Use of uninitialized value $pmean in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 155, <> line 32.
Use of uninitialized value $pstd in numeric eq (==) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 156, <> line 32.
Use of uninitialized value $af in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 156, <> line 32.
Use of uninitialized value $qual in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 157, <> line 32.
Use of uninitialized value $mapq in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 158, <> line 32.
Use of uninitialized value $af in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 158, <> line 32.
Use of uninitialized value $sn in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 159, <> line 32.
Use of uninitialized value $nm in numeric gt (>) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 161, <> line 32.
Use of uninitialized value $msi in numeric gt (>) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 162, <> line 32.
Use of uninitialized value $msi in numeric ge (>=) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 162, <> line 32.
Use of uninitialized value $hiaf in numeric lt (<) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 165, <> line 32.
Use of uninitialized value $bias in string eq at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 165, <> line 32.
Use of uninitialized value in subtraction (-) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 169, <> line 32.
Use of uninitialized value in subtraction (-) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 169, <> line 32.
Use of uninitialized value $msilen in numeric eq (==) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 169, <> line 32.
Use of uninitialized value $hiaf in numeric le (<=) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 170, <> line 32.
Use of uninitialized value $msi in numeric ge (>=) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 170, <> line 32.
Use of uninitialized value $hiaf in numeric le (<=) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 172, <> line 32.
Use of uninitialized value $msi in numeric ge (>=) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 172, <> line 32.
Use of uninitialized value $bias in substitution (s///) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 184, <> line 32.
Use of uninitialized value $vd in string le at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 185, <> line 32.
Use of uninitialized value $alt in pattern match (m//) at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 194, <> line 32.
Use of uninitialized value $ref in string eq at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 211, <> line 32.
Use of uninitialized value $alt in string eq at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 211, <> line 32.
Use of uninitialized value $vd in string ne at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 224, <> line 32.
Use of uninitialized value $vd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 225, <> line 32.
Use of uninitialized value $start in join or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $ref in join or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $dp in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $vd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $af in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $bias in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $rfwd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $rrev in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $vfwd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $vrev in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $pmean in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $pstd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $qual in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $qstd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $sbf in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $oddratio in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $mapq in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $sn in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $hiaf in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $adjaf in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $shift3 in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $msi in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $msilen in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $nm in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $hicnt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $hicov in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $lseq in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $rseq in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $dp in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $vd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $af in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $rfwd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $rrev in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $vfwd in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $vrev in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 231, <> line 32.
Use of uninitialized value $chrt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $start in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $end in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $ref in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $alt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $chrt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $start in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $end in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $ref in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $alt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $chrt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $start in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $end in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $ref in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $alt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $chrt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $start in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $end in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $ref in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
Use of uninitialized value $alt in concatenation (.) or string at /RED/RESOURCES/bcbio/anaconda/bin/var2vcf_valid.pl line 137, <> line 32.
[W::vcf_parse] Contig '' is not defined in the header. (Quick workaround: index the file with tabix.)
Encountered error, cannot proceed. Please check the error output above.
[E::bcf_write] Broken VCF record, the number of columns at chrY:26641147 does not match the number of samples (0 vs 1)
' returned non-zero exit status 255.

My config is this:

details:
- algorithm:
    aligner: bwa
    effects: vep
    ensemble:
      numpass: 2
      use_filtered: false
    mark_duplicates: true
    realign: false
    recalibrate: false
    save_diskspace: true
    tools_on:
    - vep_splicesite_annotations
    - noalt_calling
    variantcaller:
    - gatk-haplotype
    - strelka2
    - vardict
  analysis: variant2
  description: 1781F
  files:
  - /mnt/36642bae-9ec9-4100-88a2-ac173a20ea16/WORKDIR/BRAZILIANS/1781F/input/SRR6701499_1.fastq.gz
  - /mnt/36642bae-9ec9-4100-88a2-ac173a20ea16/WORKDIR/BRAZILIANS/1781F/input/SRR6701499_2.fastq.gz
  genome_build: hg38
  metadata:
    sex: female
fc_name: 1781F
upload:
  dir: ../final

This error repeated many many times in the log. Any thoughts?

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 18 (3 by maintainers)

Most upvoted comments

It is also hg19 whereas your config above specifies hg38?