Dependency Prediction of Long-Time Resource Uses in HPC Environment
High-Performance computing provides a new infrastructure for scientific calculation and its simulation. However, unbalanced load distribution among the processors causes a decreased performance in such computations, and creates a massive requirement of computing resource allocation, that requires an...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10352118/ |
| _version_ | 1797376384960561152 |
|---|---|
| author | Navin Mani Upadhyay Ravi Shankar Singh Shri Prakash Dwivedi |
| author_facet | Navin Mani Upadhyay Ravi Shankar Singh Shri Prakash Dwivedi |
| author_sort | Navin Mani Upadhyay |
| collection | DOAJ |
| description | High-Performance computing provides a new infrastructure for scientific calculation and its simulation. However, unbalanced load distribution among the processors causes a decreased performance in such computations, and creates a massive requirement of computing resource allocation, that requires an increased simulation. Therefore long-range resource utilization prediction becomes essential to achieve optimal performance in an HPC environment. This paper introduces a novel ensemble technique, which includes two algorithms, the Feature-based capability prediction algorithm(FBCA), and the Accuracy and Relative Runtime Error Prediction Algorithm (ARRE). A three-level architectural framework (the simulation environment, resource prediction, and resource queue) has also been proposed and tested on Phold and SoS. The proposed framework can deal with the requirements of computing and simulations. The FBCA algorithm reduces the redundancy between available features, and the ARRE algorithm ensures our ensemble technique’s effectiveness. We have compared the performance of the proposed schemes with other existing methods such as the Regressive Approach, Linear Regression and Random Forest, and found that our proposed algorithm achieves better accuracy from 8% to 18%. |
| first_indexed | 2024-03-08T19:37:47Z |
| format | Article |
| id | doaj.art-e8104eb6963c476f8d0f90f79e92ae2f |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T19:37:47Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e8104eb6963c476f8d0f90f79e92ae2f2023-12-26T00:07:46ZengIEEEIEEE Access2169-35362023-01-011114187114188810.1109/ACCESS.2023.334104610352118Dependency Prediction of Long-Time Resource Uses in HPC EnvironmentNavin Mani Upadhyay0https://orcid.org/0000-0003-2295-6743Ravi Shankar Singh1https://orcid.org/0000-0002-5394-7551Shri Prakash Dwivedi2https://orcid.org/0000-0002-7810-0859Department of Computer Science and Engineering, Indian Institute of Technology (BHU) Varanasi, Varanasi, Uttar Pradesh, IndiaDepartment of Computer Science and Engineering, Indian Institute of Technology (BHU) Varanasi, Varanasi, Uttar Pradesh, IndiaDepartment of Information Technology, G. B. Pant University of Agriculture and Technology, Pantnagar, IndiaHigh-Performance computing provides a new infrastructure for scientific calculation and its simulation. However, unbalanced load distribution among the processors causes a decreased performance in such computations, and creates a massive requirement of computing resource allocation, that requires an increased simulation. Therefore long-range resource utilization prediction becomes essential to achieve optimal performance in an HPC environment. This paper introduces a novel ensemble technique, which includes two algorithms, the Feature-based capability prediction algorithm(FBCA), and the Accuracy and Relative Runtime Error Prediction Algorithm (ARRE). A three-level architectural framework (the simulation environment, resource prediction, and resource queue) has also been proposed and tested on Phold and SoS. The proposed framework can deal with the requirements of computing and simulations. The FBCA algorithm reduces the redundancy between available features, and the ARRE algorithm ensures our ensemble technique’s effectiveness. We have compared the performance of the proposed schemes with other existing methods such as the Regressive Approach, Linear Regression and Random Forest, and found that our proposed algorithm achieves better accuracy from 8% to 18%.https://ieeexplore.ieee.org/document/10352118/Multi-core processorsresource predictionsocial opinion systemhigh-performance computingparallel and discrete simulation environment |
| spellingShingle | Navin Mani Upadhyay Ravi Shankar Singh Shri Prakash Dwivedi Dependency Prediction of Long-Time Resource Uses in HPC Environment IEEE Access Multi-core processors resource prediction social opinion system high-performance computing parallel and discrete simulation environment |
| title | Dependency Prediction of Long-Time Resource Uses in HPC Environment |
| title_full | Dependency Prediction of Long-Time Resource Uses in HPC Environment |
| title_fullStr | Dependency Prediction of Long-Time Resource Uses in HPC Environment |
| title_full_unstemmed | Dependency Prediction of Long-Time Resource Uses in HPC Environment |
| title_short | Dependency Prediction of Long-Time Resource Uses in HPC Environment |
| title_sort | dependency prediction of long time resource uses in hpc environment |
| topic | Multi-core processors resource prediction social opinion system high-performance computing parallel and discrete simulation environment |
| url | https://ieeexplore.ieee.org/document/10352118/ |
| work_keys_str_mv | AT navinmaniupadhyay dependencypredictionoflongtimeresourceusesinhpcenvironment AT ravishankarsingh dependencypredictionoflongtimeresourceusesinhpcenvironment AT shriprakashdwivedi dependencypredictionoflongtimeresourceusesinhpcenvironment |