Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy
As the data becomes bigger and more complex, people tend to process it in a distributed system implemented on clusters. Due to the power consumption, cost, and differentiated price-performance, the clusters are evolving into the system with heterogeneous hardware leading to the performance differenc...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8811465/ |
| _version_ | 1818557552443523072 |
|---|---|
| author | Hang Shi Yue Zhao Bofeng Zhang Kenji Yoshigoe Furong Chang |
| author_facet | Hang Shi Yue Zhao Bofeng Zhang Kenji Yoshigoe Furong Chang |
| author_sort | Hang Shi |
| collection | DOAJ |
| description | As the data becomes bigger and more complex, people tend to process it in a distributed system implemented on clusters. Due to the power consumption, cost, and differentiated price-performance, the clusters are evolving into the system with heterogeneous hardware leading to the performance difference among the nodes. Even in a homogeneous cluster, the performance of the nodes is different due to the resource competition and the communication cost. Some nodes with poor performance will drag down the efficiency of the whole system. Existing parallel computing strategies such as bulk synchronous parallel strategy and stale synchronous parallel strategy are not well suited to this problem. To address it, we proposed a free stale synchronous parallel (FSSP) strategy to free the system from the negative impact of those nodes. FSSP is improved from stale synchronous parallel (SSP) strategy, which can effectively and accurately figure out the slow nodes and eliminate the negative effects of those nodes. We validated the performance of the FSSP strategy by using some classical machine learning algorithms and datasets. Our experimental results demonstrated that FSSP was 1.5-12× faster than the bulk synchronous parallel strategy and stale synchronous parallel strategy, and it used 4× fewer iterations than the asynchronous parallel strategy to converge. |
| first_indexed | 2024-12-14T00:01:01Z |
| format | Article |
| id | doaj.art-71237daac59c434c87f7a66cba2d7c47 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T00:01:01Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-71237daac59c434c87f7a66cba2d7c472022-12-21T23:26:19ZengIEEEIEEE Access2169-35362019-01-01711876411877510.1109/ACCESS.2019.29368208811465Effective Parallel Computing via a Free Stale Synchronous Parallel StrategyHang Shi0Yue Zhao1Bofeng Zhang2Kenji Yoshigoe3Furong Chang4https://orcid.org/0000-0002-1558-4120School of Computer Engineering and Science, Shanghai University, Shanghai, ChinaSchool of Computer Engineering and Science, Shanghai University, Shanghai, ChinaSchool of Computer Engineering and Science, Shanghai University, Shanghai, ChinaFaculty of Information Networking for Innovation and Design (INIAD), Toyo University, Tokyo, JapanSchool of Computer Engineering and Science, Shanghai University, Shanghai, ChinaAs the data becomes bigger and more complex, people tend to process it in a distributed system implemented on clusters. Due to the power consumption, cost, and differentiated price-performance, the clusters are evolving into the system with heterogeneous hardware leading to the performance difference among the nodes. Even in a homogeneous cluster, the performance of the nodes is different due to the resource competition and the communication cost. Some nodes with poor performance will drag down the efficiency of the whole system. Existing parallel computing strategies such as bulk synchronous parallel strategy and stale synchronous parallel strategy are not well suited to this problem. To address it, we proposed a free stale synchronous parallel (FSSP) strategy to free the system from the negative impact of those nodes. FSSP is improved from stale synchronous parallel (SSP) strategy, which can effectively and accurately figure out the slow nodes and eliminate the negative effects of those nodes. We validated the performance of the FSSP strategy by using some classical machine learning algorithms and datasets. Our experimental results demonstrated that FSSP was 1.5-12× faster than the bulk synchronous parallel strategy and stale synchronous parallel strategy, and it used 4× fewer iterations than the asynchronous parallel strategy to converge.https://ieeexplore.ieee.org/document/8811465/Stragglerparallel strategyparallel programming |
| spellingShingle | Hang Shi Yue Zhao Bofeng Zhang Kenji Yoshigoe Furong Chang Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy IEEE Access Straggler parallel strategy parallel programming |
| title | Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy |
| title_full | Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy |
| title_fullStr | Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy |
| title_full_unstemmed | Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy |
| title_short | Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy |
| title_sort | effective parallel computing via a free stale synchronous parallel strategy |
| topic | Straggler parallel strategy parallel programming |
| url | https://ieeexplore.ieee.org/document/8811465/ |
| work_keys_str_mv | AT hangshi effectiveparallelcomputingviaafreestalesynchronousparallelstrategy AT yuezhao effectiveparallelcomputingviaafreestalesynchronousparallelstrategy AT bofengzhang effectiveparallelcomputingviaafreestalesynchronousparallelstrategy AT kenjiyoshigoe effectiveparallelcomputingviaafreestalesynchronousparallelstrategy AT furongchang effectiveparallelcomputingviaafreestalesynchronousparallelstrategy |