lenstronomy: MCMC sampler does not work with MPI=False and threadCount > 1
The schwimmbad.choose_pool
method, when mpi=False
and threadCount
> 1 returns a MultiPool object that has no is_master
attribute, hence the following error :
File "/Users/aymericg/Documents/EPFL/PhD_LASTRO/Code/Forked/lenstronomy_forked/lenstronomy/Workflow/fitting_sequence.py", line 94, in fit_sequence
mcmc_output = self.mcmc(**kwargs)
File "/Users/aymericg/Documents/EPFL/PhD_LASTRO/Code/Forked/lenstronomy_forked/lenstronomy/Workflow/fitting_sequence.py", line 230, in mcmc
threadCount=threadCount, progress=progress)
File "/Users/aymericg/Documents/EPFL/PhD_LASTRO/Code/Forked/lenstronomy_forked/lenstronomy/Sampling/sampler.py", line 149, in mcmc_emcee
if pool.is_master():
AttributeError: 'MultiPool' object has no attribute 'is_master'
In addition, I see that the new schwimmbad way of dealing with pools calls the multiprocessing
module instead of the multiprocess
, as it was done before in lenstronomy, and which is more robust because of pickling issues in multiprocessing
. @ajshajib , could you please look into that since you introduced this change ?
I did not test for the PSO, but it’s likely to give the same result.
In the meantime, I will temporarily make my own fork of schwimmbad to fix that, but it’s not viable to link lenstronomy to this fork.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 20
Commits related to this issue
- set schwimmbad requirement to master assigned to issue #166 — committed to lenstronomy/lenstronomy by sibirrer 4 years ago
Ok thanks @sibirrer , I’ll prepare the changes
up to you @aymgal. It would be great if we can minimize outside dependence on ‘unstable’ development. If you implement it in lenstronomy, I would not call it a ‘hack’ if you well document the in- and outputs, its current purpose and reason for doing it there.
@ajshajib @aymgal please use the branch multithreading_mpi to suggest changes in lenstronomy.