lenstronomy: MCMC sampler does not work with MPI=False and threadCount > 1
The schwimmbad.choose_pool method, when mpi=False and threadCount > 1 returns a MultiPool object that has no is_master attribute, hence the following error :
File "/Users/aymericg/Documents/EPFL/PhD_LASTRO/Code/Forked/lenstronomy_forked/lenstronomy/Workflow/fitting_sequence.py", line 94, in fit_sequence
mcmc_output = self.mcmc(**kwargs)
File "/Users/aymericg/Documents/EPFL/PhD_LASTRO/Code/Forked/lenstronomy_forked/lenstronomy/Workflow/fitting_sequence.py", line 230, in mcmc
threadCount=threadCount, progress=progress)
File "/Users/aymericg/Documents/EPFL/PhD_LASTRO/Code/Forked/lenstronomy_forked/lenstronomy/Sampling/sampler.py", line 149, in mcmc_emcee
if pool.is_master():
AttributeError: 'MultiPool' object has no attribute 'is_master'
In addition, I see that the new schwimmbad way of dealing with pools calls the multiprocessing module instead of the multiprocess, as it was done before in lenstronomy, and which is more robust because of pickling issues in multiprocessing. @ajshajib , could you please look into that since you introduced this change ?
I did not test for the PSO, but it’s likely to give the same result.
In the meantime, I will temporarily make my own fork of schwimmbad to fix that, but it’s not viable to link lenstronomy to this fork.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 20
Commits related to this issue
- set schwimmbad requirement to master assigned to issue #166 — committed to lenstronomy/lenstronomy by sibirrer 4 years ago
Ok thanks @sibirrer , I’ll prepare the changes
up to you @aymgal. It would be great if we can minimize outside dependence on ‘unstable’ development. If you implement it in lenstronomy, I would not call it a ‘hack’ if you well document the in- and outputs, its current purpose and reason for doing it there.
@ajshajib @aymgal please use the branch multithreading_mpi to suggest changes in lenstronomy.