Hugging Peers

RDMA-Based Algorithms for Sparse Matrix Multiplication on GPUs

☆ Save
Benjamin Brock,Aydin Buluc,Katherine Yelick
Proceedings of the 38th ACM International Conference on Supercomputing (ICS 2024) 225-235 cs.DC
10 citations published

Abstract

Sparse matrix multiplication is an important kernel for large-scale graph processing and other data-intensive applications. In this paper, we implement various asynchronous, RDMA-based sparse times dense (SpMM) and sparse times sparse (SpGEMM) algorithms, evaluating their performance running in a distributed memory setting on GPUs. Our RDMA-based implementations use the NVSHMEM communication library for direct, asynchronous one-sided communication between GPUs. We compare our asynchronous implementations to state-of-the-art bulk synchronous GPU libraries as well as a CUDA-aware MPI implementation of the SUMMA algorithm. We find that asynchronous RDMA-based implementations are able to offer favorable performance compared to bulk synchronous implementations, while also allowing for the straightforward implementation of novel work stealing algorithms.

Cite this paper

@article{brock2023,
  title = {RDMA-Based Algorithms for Sparse Matrix Multiplication on GPUs},
  author = {Benjamin Brock and Aydin Buluc and Katherine Yelick},
  journal = {Proceedings of the 38th ACM International Conference on Supercomputing (ICS 2024) 225-235},
  year = {2023},
  month = {Nov},
  doi = {10.1145/3650200.3656623},
  url = {https://doi.org/10.1145/3650200.3656623},
}