Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance Modeling
The performance of collective operations has been a critical issue since the advent of Message Passing Interface (MPI). Many algorithms have been proposed for each MPI collective operation but none of them proved optimal in all situations. Different algorithms demonstrate superior performance depend...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9502598/ |
| _version_ | 1818891414110470144 |
|---|---|
| author | Emin Nuriyev Alexey Lastovetsky |
| author_facet | Emin Nuriyev Alexey Lastovetsky |
| author_sort | Emin Nuriyev |
| collection | DOAJ |
| description | The performance of collective operations has been a critical issue since the advent of Message Passing Interface (MPI). Many algorithms have been proposed for each MPI collective operation but none of them proved optimal in all situations. Different algorithms demonstrate superior performance depending on the platform, the message size, the number of processes, etc. MPI implementations perform the selection of the collective algorithm empirically, executing a simple runtime decision function. While efficient, this approach does not guarantee the optimal selection. As a more accurate but equally efficient alternative, the use of analytical performance models of collective algorithms for the selection process was proposed and studied. Unfortunately, the previous attempts in this direction have not been successful. We revisit the analytical model-based approach and propose two innovations that significantly improve the selective accuracy of analytical models: (1) We derive analytical models from the code implementing the algorithms rather than from their high-level mathematical definitions. This results in more detailed and relevant models. (2) We estimate model parameters separately for each collective algorithm and include the execution of this algorithm in the corresponding communication experiment. We experimentally demonstrate the accuracy and efficiency of our approach using Open MPI broadcast and gather algorithms and two different Grid’5000 clusters and one supercomputer. |
| first_indexed | 2024-12-19T17:40:25Z |
| format | Article |
| id | doaj.art-3728205aff604d8aa13e8ae25273bbe7 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T17:40:25Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-3728205aff604d8aa13e8ae25273bbe72022-12-21T20:12:14ZengIEEEIEEE Access2169-35362021-01-01910935510937310.1109/ACCESS.2021.31016899502598Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance ModelingEmin Nuriyev0https://orcid.org/0000-0001-7696-2650Alexey Lastovetsky1https://orcid.org/0000-0001-9460-3897School of Computer Science, University College Dublin, Dublin 4, IrelandSchool of Computer Science, University College Dublin, Dublin 4, IrelandThe performance of collective operations has been a critical issue since the advent of Message Passing Interface (MPI). Many algorithms have been proposed for each MPI collective operation but none of them proved optimal in all situations. Different algorithms demonstrate superior performance depending on the platform, the message size, the number of processes, etc. MPI implementations perform the selection of the collective algorithm empirically, executing a simple runtime decision function. While efficient, this approach does not guarantee the optimal selection. As a more accurate but equally efficient alternative, the use of analytical performance models of collective algorithms for the selection process was proposed and studied. Unfortunately, the previous attempts in this direction have not been successful. We revisit the analytical model-based approach and propose two innovations that significantly improve the selective accuracy of analytical models: (1) We derive analytical models from the code implementing the algorithms rather than from their high-level mathematical definitions. This results in more detailed and relevant models. (2) We estimate model parameters separately for each collective algorithm and include the execution of this algorithm in the corresponding communication experiment. We experimentally demonstrate the accuracy and efficiency of our approach using Open MPI broadcast and gather algorithms and two different Grid’5000 clusters and one supercomputer.https://ieeexplore.ieee.org/document/9502598/Message passingcollective communication algorithmscommunication performance modelingMPI |
| spellingShingle | Emin Nuriyev Alexey Lastovetsky Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance Modeling IEEE Access Message passing collective communication algorithms communication performance modeling MPI |
| title | Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance Modeling |
| title_full | Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance Modeling |
| title_fullStr | Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance Modeling |
| title_full_unstemmed | Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance Modeling |
| title_short | Efficient and Accurate Selection of Optimal Collective Communication Algorithms Using Analytical Performance Modeling |
| title_sort | efficient and accurate selection of optimal collective communication algorithms using analytical performance modeling |
| topic | Message passing collective communication algorithms communication performance modeling MPI |
| url | https://ieeexplore.ieee.org/document/9502598/ |
| work_keys_str_mv | AT eminnuriyev efficientandaccurateselectionofoptimalcollectivecommunicationalgorithmsusinganalyticalperformancemodeling AT alexeylastovetsky efficientandaccurateselectionofoptimalcollectivecommunicationalgorithmsusinganalyticalperformancemodeling |