Corrfunc: script hangs when using unbuffered output

General information

  • Corrfunc version: 2.4.0
  • platform: cluster with CentOS 7.4.1708
  • installation method (pip/source/other?): first pip, then source

Issue description

i’m running Corrfunc on a simulation snapshot: i’m computing the angular correlation function in thin shells using DDtheta_mocks. I first installed Corrfunc via pip, then, in order to increase performance, via source: $ git clone https://github.com/manodeep/Corrfunc/ $ make $ make install $ python -m pip install . --user $ make tests

However, while with the pip install the code would parallelize on multiple threads, now it runs mostly on one thread. I’m submitting my SLURM job via: #!/bin/bash #SBATCH --ntasks=1 #SBATCH --nodes=1 #SBATCH --mem=480000 #SBATCH --exclusive […] export OMP_NUM_THREADS=48 srun -n 1 python -u $PY_CODE > $LOGS

Expected behavior

To run on 48 threads at ~100%.

Actual behavior

To mostly run on 1 thread. I checked with htop.

What have you tried so far?

To re-install it from source.

Minimal failing example

i’m attaching the log file corrfunc-logs.txt, including the ‘make tests’.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 17 (1 by maintainers)

Commits related to this issue

Most upvoted comments

hi @lgarrison, it works! Now the verbose output is printed into the slurm job’s standard error.

Regarding binutils/AVX-512, is it necessary to create e new environment? also, shouldn’t there be a make before pip in #196 (comment)?