Awesome
MPI bindings for Rust
The Message Passing Interface (MPI) is a specification for a message-passing style concurrency library. Implementations of MPI are often used to structure parallel computation on High Performance Computing systems. The MPI specification describes bindings for the C programming language (and through it C++) as well as for the Fortran programming language. This library tries to bridge the gap into a more rustic world.
Requirements
An implementation of the C language interface that conforms to MPI-3.1. rsmpi
is currently tested with these implementations:
- OpenMPI 4.0.3 on Ubuntu-20.04, 4.1.2 on macOS
- MPICH 3.3.2 on Ubuntu 20.04
- MS-MPI (Windows) 10.1.2 on Windows 2022
Users have also had success with these MPI implementations, but they are not tested in CI:
- Spectrum MPI 10.3.0.1
- Cray MPI 8.1.16 with
PrgEnv-amd/8.3.3
For a reasonable chance of success with rsmpi
with any MPI implementation, you must have one of:
- export
MPI_PKG_CONFIG
to be the name or path for pkg-config for your implementationrsmpi
automatically usesCRAY_MPICH_DIR
on Cray environments so the above need not be setmpich
andompi
are tried by default as a last resort- Tip: test with a command like
pkg-config --cflags --libs mpich
- The implementation provides a C compiler wrapper
mpicc
- export
MPICC=/path/to/mpicc
to specify fully - otherwise tries
mpicc
in$PATH
mpicc -show
should print the full command line that is used to invoke the wrapped C compiler in gcc-compatible syntax (e.g.,-lmpi
,-I/usr/local/include
, ...)
- export
- On Windows, the variables
MSMPI_INC
and eitherMSMPI_LIB32
orMSMPI_LIB64
should be set- see example in GitHub Actions
Since the MPI standard leaves some details of the C API unspecified (whether to implement certain constants and even functions using preprocessor macros or native C constructs, the details of most types, etc.) rsmpi
takes a two step approach to generating functional low-level bindings.
First, it uses a thin static library written in C (see rsmpi.h and rsmpi.c) that tries to capture the underspecified identifiers and re-exports them with a fixed C API. This library is built from build.rs using the gcc
crate.
Second, to generate FFI definitions tailored to each MPI implementation, rsmpi
uses rust-bindgen
which needs libclang
. See the bindgen project page for more information.
Furthermore, rsmpi
uses the libffi
crate which installs the native libffi
which depends on certain build tools. See the libffi project page for more information.
Usage
Add the mpi
crate as a dependency in your Cargo.toml
:
# "features" is optional
[dependencies]
mpi = { version = "0.8.0", features = ["user-operations", "derive"] }
Then use it in your program like this:
use mpi::request::WaitGuard;
use mpi::traits::*;
fn main() {
let universe = mpi::initialize().unwrap();
let world = universe.world();
let size = world.size();
let rank = world.rank();
let next_rank = (rank + 1) % size;
let previous_rank = (rank - 1 + size) % size;
let msg = vec![rank, 2 * rank, 4 * rank];
mpi::request::scope(|scope| {
let _sreq = WaitGuard::from(
world
.process_at_rank(next_rank)
.immediate_send(scope, &msg[..]),
);
let (msg, status) = world.any_process().receive_vec();
println!(
"Process {} got message {:?}.\nStatus is: {:?}",
rank, msg, status
);
let x = status.source_rank();
assert_eq!(x, previous_rank);
assert_eq!(vec![x, 2 * x, 4 * x], msg);
let root_rank = 0;
let root_process = world.process_at_rank(root_rank);
let mut a;
if world.rank() == root_rank {
a = vec![2, 4, 8, 16];
println!("Root broadcasting value: {:?}.", &a[..]);
} else {
a = vec![0; 4];
}
root_process.broadcast_into(&mut a[..]);
println!("Rank {} received value: {:?}.", world.rank(), &a[..]);
assert_eq!(&a[..], &[2, 4, 8, 16]);
});
}
Features
The bindings follow the MPI 3.1 specification.
Currently supported:
- Groups, Contexts, Communicators:
- Group and (Intra-)Communicator management from section 6 is mostly complete.
- no Inter-Communicators
- no process topologies
- Point to point communication:
- standard, buffered, synchronous and ready mode send in blocking and non-blocking variants
- receive in blocking and non-blocking variants
- send-receive
- probe
- matched probe/receive
- Collective communication:
- barrier
- broadcast
- (all) gather
- scatter
- all to all
- varying counts operations
- reductions/scans
- blocking and non-blocking variants
- Datatypes: Bridging between Rust types and MPI basic types as well as custom MPI datatypes which can act as views into buffers.
Not supported (yet):
- One-sided communication (RMA)
- MPI parallel I/O
- A million small things
Optional Cargo Features
These optional features can be enabled in your cargo manifest. See the Usage section above.
user-operations
enables capturing lambdas and safe creation in UserOperation
. This feature
requires the libffi
system library, which is not available on all systems out-of-the-box.
let mut h = 0;
comm.all_reduce_into(
&(rank + 1),
&mut h,
&UserOperation::commutative(|x, y| {
let x: &[Rank] = x.downcast().unwrap();
let y: &mut [Rank] = y.downcast().unwrap();
for (&x_i, y_i) in x.iter().zip(y) {
*y_i += x_i;
}
}),
);
derive
enables the Equivalence
derive macro, which makes it easy to send structs
over-the-wire without worrying about safety around padding, and allowing arbitrary datatype
matching between structs with the same field order but different layout.
#[derive(Equivalence)]
struct MyProgramOpts {
name: [u8; 100],
num_cycles: u32,
material_properties: [f64; 20],
}
Documentation
Every public item of rsmpi
should at least have a short piece of documentation associated with it. Documentation can be generated via:
cargo doc
Documentation for the latest version of the crate released to crates.io is hosted on Github pages.
Examples
See files in examples/. These examples also act as integration tests.
Python integration
It is possible to use rsmpi
with a communicator provided by mpi4py. An example project demonstrating this is mpi4py_with_rsmpi.
License
Licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.