Accelerated Synchronous Model Parallelism Using Cooperative Process for Training Compute-Intensive Models
As deep learning has been recently applied to a wide variety of fields, there is a growing demand for models that can handle large input data, including high-resolution images. Therefore, model parallelism was proposed to train a model whose size exceeds the memory capacity of an accelerator, but th...
Main Authors: | Chanhee Yu, Kyongseok Park |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2023-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/10185201/ |
Similar Items
-
Control Method of Parallel Inverters with Self-Synchronizing Characteristics in Distributed Microgrid
by: Xiangwu Yan, et al.
Published: (2019-10-01) -
A Pre-Synchronization Method for Parallel VSGs of Distributed Microgrid Based on Control Mode Switching
by: Kunyu Dong, et al.
Published: (2023-05-01) -
Design and Evaluation of the Hamal Parallel Computer
by: Grossman, J.P.
Published: (2004) -
<tt>SHAT</tt>: A Novel Asynchronous Training Algorithm That Provides Fast Model Convergence in Distributed Deep Learning
by: Yunyong Ko, et al.
Published: (2021-12-01) -
Design and parameter analysis of an improved pre-synchronization method for multiple inverters based on virtual synchronization generator control in microgrid
by: Liang Meng, et al.
Published: (2022-08-01)