A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques
This paper presents a selective review of statistical computation methods for massive data analysis. A huge amount of statistical methods for massive data computation have been rapidly developed in the past decades. In this work, we focus on three categories of statistical computation methods: (1) d...
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2024-04-01
|
| Series: | Statistical Theory and Related Fields |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/24754269.2024.2343151 |
| _version_ | 1797193654151938048 |
|---|---|
| author | Xuetong Li Yuan Gao Hong Chang Danyang Huang Yingying Ma Rui Pan Haobo Qi Feifei Wang Shuyuan Wu Ke Xu Jing Zhou Xuening Zhu Yingqiu Zhu Hansheng Wang |
| author_facet | Xuetong Li Yuan Gao Hong Chang Danyang Huang Yingying Ma Rui Pan Haobo Qi Feifei Wang Shuyuan Wu Ke Xu Jing Zhou Xuening Zhu Yingqiu Zhu Hansheng Wang |
| author_sort | Xuetong Li |
| collection | DOAJ |
| description | This paper presents a selective review of statistical computation methods for massive data analysis. A huge amount of statistical methods for massive data computation have been rapidly developed in the past decades. In this work, we focus on three categories of statistical computation methods: (1) distributed computing, (2) subsampling methods, and (3) minibatch gradient techniques. The first class of literature is about distributed computing and focuses on the situation, where the dataset size is too huge to be comfortably handled by one single computer. In this case, a distributed computation system with multiple computers has to be utilized. The second class of literature is about subsampling methods and concerns about the situation, where the blacksample size of dataset is small enough to be placed on one single computer but too large to be easily processed by its memory as a whole. The last class of literature studies those minibatch gradient related optimization techniques, which have been extensively used for optimizing various deep learning models. |
| first_indexed | 2024-04-24T05:43:49Z |
| format | Article |
| id | doaj.art-88f77d041db947be8d00ebfb9ea8dcc2 |
| institution | Directory Open Access Journal |
| issn | 2475-4269 2475-4277 |
| language | English |
| last_indexed | 2024-04-24T05:43:49Z |
| publishDate | 2024-04-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Statistical Theory and Related Fields |
| spelling | doaj.art-88f77d041db947be8d00ebfb9ea8dcc22024-04-23T18:43:37ZengTaylor & Francis GroupStatistical Theory and Related Fields2475-42692475-42772024-04-0112310.1080/24754269.2024.2343151A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniquesXuetong Li0Yuan Gao1Hong Chang2Danyang Huang3Yingying Ma4Rui Pan5Haobo Qi6Feifei Wang7Shuyuan Wu8Ke Xu9Jing Zhou10Xuening Zhu11Yingqiu Zhu12Hansheng Wang13Guanghua School of Management, Peking University, Beijing, People's Republic of ChinaGuanghua School of Management, Peking University, Beijing, People's Republic of ChinaGuanghua School of Management, Peking University, Beijing, People's Republic of ChinaCenter for Applied Statistics and School of Statistics, Renmin University of China, Beijing, People's Republic of ChinaSchool of Economics and Management, Beihang University, Beijing, People's Republic of ChinaSchool of Statistics and Mathematics, Central University of Finance and Economics, Beijing, People's Republic of ChinaSchool of Statistics, Beijing Normal University, Beijing, People's Republic of ChinaCenter for Applied Statistics and School of Statistics, Renmin University of China, Beijing, People's Republic of ChinaSchool of Statistics and Management, Shanghai University of Finance and Economics, Shanghai, People's Republic of ChinaSchool of Statistics, University of International Business and Economics, Beijing, People's Republic of ChinaCenter for Applied Statistics and School of Statistics, Renmin University of China, Beijing, People's Republic of ChinaSchool of Data Science and MOE Laboratory for National Development and Intelligent Governance, Fudan University, Shanghai, People's Republic of ChinaSchool of Statistics, University of International Business and Economics, Beijing, People's Republic of ChinaGuanghua School of Management, Peking University, Beijing, People's Republic of ChinaThis paper presents a selective review of statistical computation methods for massive data analysis. A huge amount of statistical methods for massive data computation have been rapidly developed in the past decades. In this work, we focus on three categories of statistical computation methods: (1) distributed computing, (2) subsampling methods, and (3) minibatch gradient techniques. The first class of literature is about distributed computing and focuses on the situation, where the dataset size is too huge to be comfortably handled by one single computer. In this case, a distributed computation system with multiple computers has to be utilized. The second class of literature is about subsampling methods and concerns about the situation, where the blacksample size of dataset is small enough to be placed on one single computer but too large to be easily processed by its memory as a whole. The last class of literature studies those minibatch gradient related optimization techniques, which have been extensively used for optimizing various deep learning models.https://www.tandfonline.com/doi/10.1080/24754269.2024.2343151Distributed computingmassive data analysisminibatch techniquesstochastic optimizationsubsampling methods |
| spellingShingle | Xuetong Li Yuan Gao Hong Chang Danyang Huang Yingying Ma Rui Pan Haobo Qi Feifei Wang Shuyuan Wu Ke Xu Jing Zhou Xuening Zhu Yingqiu Zhu Hansheng Wang A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques Statistical Theory and Related Fields Distributed computing massive data analysis minibatch techniques stochastic optimization subsampling methods |
| title | A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques |
| title_full | A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques |
| title_fullStr | A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques |
| title_full_unstemmed | A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques |
| title_short | A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques |
| title_sort | selective review on statistical methods for massive data computation distributed computing subsampling and minibatch techniques |
| topic | Distributed computing massive data analysis minibatch techniques stochastic optimization subsampling methods |
| url | https://www.tandfonline.com/doi/10.1080/24754269.2024.2343151 |
| work_keys_str_mv | AT xuetongli aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT yuangao aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT hongchang aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT danyanghuang aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT yingyingma aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT ruipan aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT haoboqi aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT feifeiwang aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT shuyuanwu aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT kexu aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT jingzhou aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT xueningzhu aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT yingqiuzhu aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT hanshengwang aselectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT xuetongli selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT yuangao selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT hongchang selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT danyanghuang selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT yingyingma selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT ruipan selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT haoboqi selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT feifeiwang selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT shuyuanwu selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT kexu selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT jingzhou selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT xueningzhu selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT yingqiuzhu selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques AT hanshengwang selectivereviewonstatisticalmethodsformassivedatacomputationdistributedcomputingsubsamplingandminibatchtechniques |