qsiprep: qsirecon error in 0.19.1: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 4095: unexpected end of data

I’m possibly doing something wrong here, but I can’t figure out why I see this error.

I’m calling qsirecon with this spec

{
  "name": "dsistudio_pipeline",
  "space": "T1w",
  "atlases": ["schaefer100", "schaefer200", "schaefer400", "brainnetome246", "aicha384", "gordon333", "aal116"],
  "nodes": [
    {
      "name": "dsistudio_gqi",
      "software": "DSI Studio",
      "action": "reconstruction",
      "input": "qsiprep",
      "output_suffix": "gqi",
      "parameters": {"method": "gqi"}
    },
    {
      "name": "scalar_export",
      "software": "DSI Studio",
      "action": "export",
      "input": "dsistudio_gqi",
      "output_suffix": "gqiscalar"
    },
    {
      "name": "tractography",
      "software": "DSI Studio",
      "action": "tractography",
      "input": "dsistudio_gqi",
      "parameters": {
        "turning_angle": 35,
        "smoothing": 0.0,
        "step_size": 1.0,
        "min_length": 30,
        "max_length": 250,
        "seed_plan": 0,
        "interpolation": 0,
        "initial_dir": 2,
        "fiber_count": 5000000
      }
    },
    {
      "name": "streamline_connectivity",
      "software": "DSI Studio",
      "action": "connectivity",
      "input": "tractography",
      "output_suffix": "gqinetwork",
      "parameters": {
        "connectivity_value": "count,ncount,mean_length,gfa",
        "connectivity_type": "pass,end"
      }
    }
  ]
}

taken from the repository file under qsiprep/data/pipelines/dsi_studio_gqi.json

The runscript is

module load singularity

image=/project/ftdc_pipeline/ftdc-picsl/pmacsPreps-0.2.3/containers/qsiprep-0.19.1.sif

export SINGULARITYENV_TMPDIR=/tmp

tmpDir=`mktemp -d -p /scratch/pac`

echo "Using tmp dir $tmpDir"

singularity run \
  --containall \
  -B ${tmpDir}:/tmp,/appl/freesurfer-7.1.1/license.txt:/freesurfer/license.txt \
  -B $PWD/output_old:/data/output,/project/ftdc_misc/harmonization/analysis/synb0/bids:/data/bids,${PWD}/dsi_studio_gqi.json:/recon/spec.json \
  $image /data/bids /data/output participant \
  --participant_label HM01 \
  --recon-spec /recon/spec.json \
  --recon_input /data/output \
  --recon-only \
  --nthreads 1 \
  --omp-nthreads 1 \
  --work-dir /tmp \
  --fs-license-file /freesurfer/license.txt

It gets through the tractography stage but fails on connectivity with

231201-12:20:28,664 nipype.workflow INFO:
	 [Node] Finished "gordon333", elapsed time 32.833375s.
231201-12:20:28,664 nipype.workflow WARNING:
	 Storing result file without outputs
231201-12:20:28,666 nipype.workflow WARNING:
	 [Node] Error on "dsistudio_atlasgraph.gordon333" (/tmp/qsirecon_wf/sub-HM01_dsistudio_pipeline/sub_HM01_ses_SC3Tx20231019x1312_acq_MultiShellDir117_space_T1w_desc_preproc_recon_wf/streamline_connectivity/calc_connectivity/dsistudio_atlasgraph/gordon333)
231201-12:20:28,671 nipype.workflow ERROR:
	 Node gordon333 failed to run on host ftdc-gpu01.
231201-12:20:28,675 nipype.workflow ERROR:
	 Saving crash info to /home/cookpa/crash-20231201-122028-cookpa-gordon333-e4aa145f-b0aa-41af-9094-de25420677f1.pklz
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/linear.py", line 47, in run
    node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node gordon333.

Cmdline:
	dsi_studio --action=ana  --connectivity=/tmp/qsirecon_wf/sub-HM01_dsistudio_pipeline/sub_HM01_ses_SC3Tx20231019x1312_acq_MultiShellDir117_space_T1w_desc_preproc_anat_wf/get_atlases/tpl-MNI152NLin2009cAsym_res-01_atlas-Gordon333Ext_desc-LPS_dseg_to_dwi.nii.gz --connectivity_type=pass,end --connectivity_value=count,ncount,mean_length,gfa --source=/tmp/qsirecon_wf/sub-HM01_dsistudio_pipeline/sub_HM01_ses_SC3Tx20231019x1312_acq_MultiShellDir117_space_T1w_desc_preproc_recon_wf/streamline_connectivity/calc_connectivity/dsistudio_atlasgraph/gordon333/sub-HM01_ses-SC3Tx20231019x1312_acq-MultiShellDir117_space-T1w_desc-preproc_dwi.src.gz.odf.gqi.1.25.fib.gz --thread_count=1 --thread_count=1 --tract=/tmp/qsirecon_wf/sub-HM01_dsistudio_pipeline/sub_HM01_ses_SC3Tx20231019x1312_acq_MultiShellDir117_space_T1w_desc_preproc_recon_wf/streamline_connectivity/calc_connectivity/dsistudio_atlasgraph/gordon333/sub-HM01_ses-SC3Tx20231019x1312_acq-MultiShellDir117_space-T1w_desc-preproc_dwi.src.gz.odf.gqi.1.25.fib.trk.gz
Stdout:

Stderr:

Traceback:
	Traceback (most recent call last):
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 764, in _run_interface
	    runtime = run_command(
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/utils/subprocess.py", line 141, in run_command
	    _process()
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/utils/subprocess.py", line 137, in _process
	    stream.read(drain)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/utils/subprocess.py", line 42, in read
	    while self._read(drain) is not None:
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/utils/subprocess.py", line 49, in _read
	    buf = os.read(fd, 4096).decode(self.default_encoding)
	UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4094-4095: unexpected end of data

I get this same error for all the atlases. I see this error with qsiprep 0.19.1 and 0.18.1, but not 0.16.1