pass custom mpi4py communicator#94
Conversation
adematti
commented
Oct 5, 2022
adematti
commented
- ability to provide, through the Python wrapper, the proper mpi4py communicator
- drop use of MPI_ANY_TAG to avoid conflict with potential user communications
There was a problem hiding this comment.
Hi @adematti, many thanks for putting this together. Apologies it has taken so long to get to review this. We're now in the process of trying to merge in a few PRs, and it would be great to include this one if you have time to look at this.
This change is actually really useful, and it looks like you've done some quite clever work to get this in place.
As submitted it was quite hard to review this PR, since your linter has made a lot of semantically irrelevant adjustments to the code, which might cause problems down the line as we try to merge in other changes. This was partly why I didn't have an opportunity to quickly review at the time. I've now reverted these.
Could you
a) check my resolutions given the latest changes to master
b) provide a bit of explanation for the semantically meaningful changes in setup.py? This is a file which has to accommodate several different setups (mac, linux etc), so should be changed with caution!
| #cc_compiler = subprocess.check_output(["make", "print_CC"]).decode('utf-8').strip() | ||
| #os.environ"CC"] = cc_compiler | ||
|
|
||
| cxx_compiler = subprocess.check_output(["make", "print_CXX"]).decode('utf-8').strip() | ||
| os.environ["CXX"] = cxx_compiler | ||
| #cxx_compiler = subprocess.check_output(["make", "print_CXX"]).decode('utf-8').strip() | ||
| #os.environ["CXX"] = cxx_compiler |
There was a problem hiding this comment.
Is there a reason these need to be commented out?
* fix errant backslash * Adjusted the setup.py to not point to ccpforge * Minor typo picked up in #94 * version bump --------- Co-authored-by: Will Handley <wh260@cam.ac.uk>
williamjameshandley
left a comment
williamjameshandley
left a comment
There was a problem hiding this comment.
@adematti I'm quite keen to get this merged, since I have a potential use-case for this myself.
Could you double check the changes I've made, and if you're happy I will merge them in.
Tests that the new comm parameter is properly supported in both run_polychord() and run() interfaces. Validates interface exists without running full MPI tests that could hang in CI. Validates PR PolyChord#94 custom communicator feature.
Tests that PolyChord actually runs successfully with: - Default COMM_WORLD behavior - Explicit COMM_WORLD parameter - Custom duplicated communicator Uses realistic 2D Gaussian likelihood and validates that the custom communicator feature works end-to-end through all Python -> C++ -> Fortran layers. Validates PR PolyChord#94 MPI communicator functionality.
2 participants
