Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling System
Synchrotron radiation sources are widely used in interdisciplinary research, generating an enormous amount of data while posing serious challenges to the storage, processing, and analysis capabilities of the large-scale scientific facilities worldwide. A flexible and scalable computing architecture,...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/9/5387 |
| _version_ | 1797603058017894400 |
|---|---|
| author | Jing Ye Chunpeng Wang Jige Chen Rongzheng Wan Xiaoyun Li Alessandro Sepe Renzhong Tai |
| author_facet | Jing Ye Chunpeng Wang Jige Chen Rongzheng Wan Xiaoyun Li Alessandro Sepe Renzhong Tai |
| author_sort | Jing Ye |
| collection | DOAJ |
| description | Synchrotron radiation sources are widely used in interdisciplinary research, generating an enormous amount of data while posing serious challenges to the storage, processing, and analysis capabilities of the large-scale scientific facilities worldwide. A flexible and scalable computing architecture, suitable for complex application scenarios, combined with efficient and intelligent scheduling strategies, plays a key role in addressing these issues. In this work, we present a novel cloud–edge hybrid intelligent system (CEHIS), which was architected, developed, and deployed by the Big Data Science Center (BDSC) at the Shanghai Synchrotron Radiation Facility (SSRF) and meets the computational needs of the large-scale scientific facilities. Our methodical simulations demonstrate that the CEHIS is more efficient and performs better than the cloud-based model. Here, we have applied a deep reinforcement learning approach to the task scheduling system, finding that it effectively reduces the total time required for the task completion. Our findings prove that the cloud–edge hybrid intelligent architectures are a viable solution to address the requirements and conditions of the modern synchrotron radiation facilities, further enhancing their data processing and analysis capabilities. |
| first_indexed | 2024-03-11T04:24:07Z |
| format | Article |
| id | doaj.art-5287dfb03f9d4dd2a150cf5275443bde |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T04:24:07Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-5287dfb03f9d4dd2a150cf5275443bde2023-11-17T22:33:20ZengMDPI AGApplied Sciences2076-34172023-04-01139538710.3390/app13095387Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling SystemJing Ye0Chunpeng Wang1Jige Chen2Rongzheng Wan3Xiaoyun Li4Alessandro Sepe5Renzhong Tai6Shanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, ChinaShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, ChinaShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, ChinaShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, ChinaShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, ChinaShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, ChinaShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, ChinaSynchrotron radiation sources are widely used in interdisciplinary research, generating an enormous amount of data while posing serious challenges to the storage, processing, and analysis capabilities of the large-scale scientific facilities worldwide. A flexible and scalable computing architecture, suitable for complex application scenarios, combined with efficient and intelligent scheduling strategies, plays a key role in addressing these issues. In this work, we present a novel cloud–edge hybrid intelligent system (CEHIS), which was architected, developed, and deployed by the Big Data Science Center (BDSC) at the Shanghai Synchrotron Radiation Facility (SSRF) and meets the computational needs of the large-scale scientific facilities. Our methodical simulations demonstrate that the CEHIS is more efficient and performs better than the cloud-based model. Here, we have applied a deep reinforcement learning approach to the task scheduling system, finding that it effectively reduces the total time required for the task completion. Our findings prove that the cloud–edge hybrid intelligent architectures are a viable solution to address the requirements and conditions of the modern synchrotron radiation facilities, further enhancing their data processing and analysis capabilities.https://www.mdpi.com/2076-3417/13/9/5387cloudedgehybrid architecturesynchrotronbig datamachine learning |
| spellingShingle | Jing Ye Chunpeng Wang Jige Chen Rongzheng Wan Xiaoyun Li Alessandro Sepe Renzhong Tai Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling System Applied Sciences cloud edge hybrid architecture synchrotron big data machine learning |
| title | Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling System |
| title_full | Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling System |
| title_fullStr | Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling System |
| title_full_unstemmed | Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling System |
| title_short | Cloud–Edge Hybrid Computing Architecture for Large-Scale Scientific Facilities Augmented with an Intelligent Scheduling System |
| title_sort | cloud edge hybrid computing architecture for large scale scientific facilities augmented with an intelligent scheduling system |
| topic | cloud edge hybrid architecture synchrotron big data machine learning |
| url | https://www.mdpi.com/2076-3417/13/9/5387 |
| work_keys_str_mv | AT jingye cloudedgehybridcomputingarchitectureforlargescalescientificfacilitiesaugmentedwithanintelligentschedulingsystem AT chunpengwang cloudedgehybridcomputingarchitectureforlargescalescientificfacilitiesaugmentedwithanintelligentschedulingsystem AT jigechen cloudedgehybridcomputingarchitectureforlargescalescientificfacilitiesaugmentedwithanintelligentschedulingsystem AT rongzhengwan cloudedgehybridcomputingarchitectureforlargescalescientificfacilitiesaugmentedwithanintelligentschedulingsystem AT xiaoyunli cloudedgehybridcomputingarchitectureforlargescalescientificfacilitiesaugmentedwithanintelligentschedulingsystem AT alessandrosepe cloudedgehybridcomputingarchitectureforlargescalescientificfacilitiesaugmentedwithanintelligentschedulingsystem AT renzhongtai cloudedgehybridcomputingarchitectureforlargescalescientificfacilitiesaugmentedwithanintelligentschedulingsystem |