atac_dnase_pipelines: Error : Creating checkpoint file

Hi,

I have been trying to run your pipeline with the mm10 data installed from install_genome_data. My output looks like something in the file “myOutStruct” compared to the structure given on the ATAC-seq gitHub page (screenshot in “originalOutStruct”)

I am running the command :

 $ bds atac.bds -species mm10 -gensz mm -bam1 

…/genomes/mm10_enh_dhs.bam -chrsz …/genomes/mm10/mm10.chrom.sizes

I have generated the bam file using the bed files within the mm10 download/install.

It also generates the error file (“task.postalign_bam.markdup_bam_picard_rep1.line_409.id_10.stderr”). Which looks something like :

Picked up _JAVA_OPTIONS: -Xms256M -Xmx12G -XX:ParallelGCThreads=1 [Wed Jun 27 17:07:04 BST 2018] picard.sam.markduplicates.MarkDuplicates INPUT=[/homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.filt.bam] OUTPUT=/homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam METRICS_FILE=/homes/reham/ATAC-Seq/atac_dnase_pipelines/out/qc/rep1/mm10_enh_dhs.dup.qc REMOVE_DUPLICATES=false ASSUME_SORTED=true VALIDATION_STRINGENCY=LENIENT MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=8000 SORTING_COLLECTION_SIZE_RATIO=0.25 PROGRAM_RECORD_ID=MarkDuplicates PROGRAM_GROUP_NAME=MarkDuplicates DUPLICATE_SCORING_STRATEGY=SUM_OF_BASE_QUALITIES READ_NAME_REGEX=[a-zA-Z0-9]+:[0-9]:([0-9]+)😦[0-9]+)😦[0-9]+).* OPTICAL_DUPLICATE_PIXEL_DISTANCE=100 VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false [Wed Jun 27 17:07:04 BST 2018] Executing as reham@hx-noah-08-10.ebi.ac.uk on Linux 3.10.0-514.16.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_131-b11; Picard version: 1.126(4691ee611ac205d4afe2a1b7a2ea975a6f997426_1417447214) IntelDeflater INFO 2018-06-27 17:07:04 MarkDuplicates Start of doWork freeMemory: 256939704; totalMemory: 259588096; maxMemory: 12455444480 INFO 2018-06-27 17:07:04 MarkDuplicates Reading input file and constructing read end information. INFO 2018-06-27 17:07:04 MarkDuplicates Will retain up to 47905555 data points before spilling to disk. INFO 2018-06-27 17:07:07 MarkDuplicates Read 0 records. 0 pairs never matched. INFO 2018-06-27 17:07:07 MarkDuplicates After buildSortedReadEndLists freeMemory: 539996048; totalMemory: 929882112; maxMemory: 12455444480 INFO 2018-06-27 17:07:07 MarkDuplicates Will retain up to 389232640 duplicate indices before spilling to disk.

The output I am presently getting concludes with the following :

Distributing 8 to ... 
{1=8}
pyenv: activate: command not found
Task failed:
	Program & line     : 'modules/postalign_bam.bds', line 341
	Task Name          : 'dedup_bam_PE_2 rep1'
	Task ID            : 'atac.bds.20180628_151527_655_parallel_41/task.postalign_bam.dedup_bam_PE_2_rep1.line_341.id_10'
	Task PID           : '4097'
	Task hint          : 'samtools view -F 1804 -f 2 -b /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam > /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.nodup.bam; sambamba index -t 8 /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.nodup.bam; s'
	Task resources     : 'cpus: 8	mem: -1.0 B	wall-timeout: 8640000'
	State              : 'ERROR'
	Dependency state   : 'ERROR'
	Retries available  : '1'
	Input files        : '[/homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam]'
	Output files       : '[/homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.nodup.bam, /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/qc/rep1/mm10_enh_dhs.nodup.flagstat.qc, /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/qc/rep1/mm10_enh_dhs.nodup.pbc.qc]'
	Script file        : '/homes/reham/ATAC-Seq/atac_dnase_pipelines/atac.bds.20180628_151527_655_parallel_41/task.postalign_bam.dedup_bam_PE_2_rep1.line_341.id_10.sh'
	Exit status        : '1'
	Program            : 
		
		# SYS command. line 343
		
		 if [[ -f $(which conda) && $(conda env list | grep bds_atac | wc -l) != "0" ]]; then source activate bds_atac; sleep 5; fi;  export PATH=/homes/reham/ATAC-Seq/atac_dnase_pipelines/.:/homes/reham/ATAC-Seq/atac_dnase_pipelines/modules:/homes/reham/ATAC-Seq/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)
		
		# SYS command. line 350
		
		 samtools view -F 1804 -f 2 -b /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam > /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.nodup.bam
		
		# SYS command. line 352
		
		 sambamba index -t 8 /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.nodup.bam
		
		# SYS command. line 354
		
		 sambamba flagstat -t 8 /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.nodup.bam > /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/qc/rep1/mm10_enh_dhs.nodup.flagstat.qc
		
		# SYS command. line 365
		
		 sambamba sort -t 8 -n /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam -o /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam.tmp.bam
		
		# SYS command. line 367
		
		 bedtools bamtobed -bedpe -i /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam.tmp.bam | \
							awk 'BEGIN{OFS="\t"}{print $1,$2,$4,$6,$9,$10}' | \
							grep -v 'chrM' | sort | uniq -c | \
							awk 'BEGIN{mt=0;m0=0;m1=0;m2=0} ($1==1){m1=m1+1} ($1==2){m2=m2+1} {m0=m0+1} {mt=mt+$1} END{m1_m2=-1.0; if(m2>0) m1_m2=m1/m2; printf "%d\t%d\t%d\t%d\t%f\t%f\t%f\n",mt,m0,m1,m2,m0/mt,m1/m0,m1_m2}' > /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/qc/rep1/mm10_enh_dhs.nodup.pbc.qc
		
		# SYS command. line 371
		
		 rm -f /homes/reham/ATAC-Seq/atac_dnase_pipelines/out/align/rep1/mm10_enh_dhs.dupmark.bam.tmp.bam
		
		# SYS command. line 373
		
		 TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; sleep 0
	StdErr (100000000 lines)  :
		pyenv: activate: command not found

Fatal error: modules/postalign_bam.bds, line 378, pos 4. Task/s failed.

Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
Fatal error: atac.bds, line 426, pos 2. Task/s failed.

Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.

Could you please help me in generating the right output ?

Many thanks. Kind Regards, Reham