fmriprep: fmriprep 1.4.1 file not found error
I am running a Singularity container of fmriprep 1.4.1. When we try to run the following command, we get a Traceback error indicating that it cannot find a file which I think it was supposed to have created itself.
The command I run is
singularity run -B /sw:/sw /tmp/fmriprep-1.4.1.simg $PWD --fs-license-file=/sw/arcts/centos7/freesurfer/6.0.0/license.txt --participant-label=sub-NDARINVZZZNB0XC output participant
I get the same traceback using that same command but with the added options --nthreads 1 --omp-nthreads 1
.
It runs through setting up the nipype nodes successfully it seems, then prints up to the ### References
, at which point it produces this output
190820-15:39:04,174 nipype.workflow INFO:
[Node] Setting-up "fmriprep_wf.single_subject_NDARINVZZZNB0XC_wf.func_preproc_ses_baselineYear1Arm1_task_mid_run_01_wf.func_derivatives_wf.raw_sources" in "/tmp/fmriprep/work/fmriprep_wf/single_subject_NDARINVZZZNB0XC_wf/func_preproc_ses_baselineYear1Arm1_task_mid_run_01_wf/func_derivatives_wf/raw_sources".
190820-15:39:04,175 nipype.workflow INFO:
[Node] Running "raw_sources" ("nipype.interfaces.utility.wrappers.Function")
exception calling callback for <Future at 0x2af37ff648d0 state=finished raised OSError>
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
result['result'] = node.run(updatehash=updatehash)
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 487, in run
self, report_type='postexec', is_mapnode=isinstance(self, MapNode))
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 149, in write_report
result = node.result # Locally cache result
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 197, in result
return _load_resultfile(self.output_dir(), self.name)[0]
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 359, in load_resultfile
basedir=path).items()):
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 478, in modify_paths
val, relative=relative, basedir=basedir)
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 484, in modify_paths
modify_paths(val, relative=relative, basedir=basedir))
File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 498, in modify_paths
raise IOError('File %s not found' % out)
OSError: File /tmp/fmriprep/work/fmriprep_wf/single_subject_NDARINVZZZNB0XC_wf/func_preproc_ses_baselineYear1Arm1_task_mid_run_01_wf/func_derivatives_wf/raw_sources/sub-NDARINVZZZNB0XC/ses-baselineYear1Arm1/func/sub-NDARINVZZZNB0XC_ses-baselineYear1Arm1_task-mid_run-01_bold.nii not found
It appears that this directory does exist: /tmp/fmriprep/work/fmriprep_wf/single_subject_NDARINVZZZNB0XC_wf/func_preproc_ses_baselineYear1Arm1_task_mid_run_01_wf/func_derivatives_wf/raw_sources
, but that there is no subject directory underneath it.
$ ls /tmp/fmriprep/work/fmriprep_wf/single_subject_NDARINVZZZNB0XC_wf/func_preproc_ses_baselineYear1Arm1_task_mid_run_01_wf/func_derivatives_wf/raw_sources
_0x763007af6eb61212a00403b1b1920887.json _inputs.pklz _node.pklz _report result_raw_sources.pklz
Under the BIDS_ROOT
directory, the path sub-NDARINVZZZNB0XC/ses-baselineYear1Arm1/func/sub-NDARINVZZZNB0XC_ses-baselineYear1Arm1_task-mid_run-01_bold.nii
.
The same subject gets a good deal past that stage when run using a Singularity container built for fmriprep-1.1.8, so I think it is not a problem with the data. Since we tried this with the options to limit threading to 1 process and 1 thread, it should not be asynchronous threading.
The system on which I am running is CentOS Linux release 7.6.1810 (Core) on a Dell R740 with dual Intel® Xeon® Gold 6140 CPU @ 2.30GHz chips. The kernel version is 3.10.0-957.10.1.el7.x86_64.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 16 (6 by maintainers)
We’ve been tearing our hair out over the same error. fmriprep 1.4.1 running in a singularity container, works on some subjects, throws the same error you got on others. I’m glad it’s not just us…
@effigies just an FYI, this issue does seem to be very related to issue #1758. I tried launching fmriprep in the same way as in the end of that thread, but without using the workaround.yml file and it does seem to get beyond the raw_sources step quickly.
Define “slowly”? We have no progress after 20 hours.
Yes, we were prematurely jubilant, because we got past our initial blocker. We ran into the same problem.