Ensemble Method of Diverse Regularized Extreme Learning Machines
As a fast training algorithm of single hidden layer forward networks, extreme learning machine (ELM) randomly initializes the input layer weights and hidden layer biases, and gets the weights of output layer through the analysis method. It overcomes many shortcomings of gradient based learning algor...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press
2022-08-01
|
| Series: | Jisuanji kexue yu tansuo |
| Subjects: | |
| Online Access: | http://fcst.ceaj.org/fileup/1673-9418/PDF/2101001.pdf |
| _version_ | 1818000573363912704 |
|---|---|
| author | CHEN Yang, WANG Shitong |
| author_facet | CHEN Yang, WANG Shitong |
| author_sort | CHEN Yang, WANG Shitong |
| collection | DOAJ |
| description | As a fast training algorithm of single hidden layer forward networks, extreme learning machine (ELM) randomly initializes the input layer weights and hidden layer biases, and gets the weights of output layer through the analysis method. It overcomes many shortcomings of gradient based learning algorithm, such as local minimum, inappropriate learning rate, slow learning speed, etc. However, ELM still inevitably has overfitting and poorly stable phenomenon, especially on large-scale datasets. This paper proposes the ensemble method of diverse regularized extreme learning machines (DRELM) to solve the above problems. First, its own random distribution weigthts are used to assure the diversity between each ELM base learner, then leave-one-out (LOO) cross validation method and M S E P R E S Smethod are used to find the optimal hidden node number of each base learner, calculate the optimal hidden layer output weights to train better and different base learners. Then the new penalty term about diversity is explicitly added to the objective function and the output matrix of each learner is updated iteratively. Finally, the final output of the whole network model is obtained by averaging the output of all base learners. This method can effectively realize the ensemble of regularized extreme learning machines (RELM) with both accuracy and diversity. Experimental results on 10 UCI datasets indicate the effectiveness of DRELM. |
| first_indexed | 2024-04-14T03:23:12Z |
| format | Article |
| id | doaj.art-fdb336931b064523ba1c06ef750b1311 |
| institution | Directory Open Access Journal |
| issn | 1673-9418 |
| language | zho |
| last_indexed | 2024-04-14T03:23:12Z |
| publishDate | 2022-08-01 |
| publisher | Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press |
| record_format | Article |
| series | Jisuanji kexue yu tansuo |
| spelling | doaj.art-fdb336931b064523ba1c06ef750b13112022-12-22T02:15:15ZzhoJournal of Computer Engineering and Applications Beijing Co., Ltd., Science PressJisuanji kexue yu tansuo1673-94182022-08-011681819182810.3778/j.issn.1673-9418.2101001Ensemble Method of Diverse Regularized Extreme Learning MachinesCHEN Yang, WANG Shitong01. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122, China;2. Key Laboratory of Media Design and Software Technology of Jiangsu Province, Jiangnan University, Wuxi, Jiangsu 214122, ChinaAs a fast training algorithm of single hidden layer forward networks, extreme learning machine (ELM) randomly initializes the input layer weights and hidden layer biases, and gets the weights of output layer through the analysis method. It overcomes many shortcomings of gradient based learning algorithm, such as local minimum, inappropriate learning rate, slow learning speed, etc. However, ELM still inevitably has overfitting and poorly stable phenomenon, especially on large-scale datasets. This paper proposes the ensemble method of diverse regularized extreme learning machines (DRELM) to solve the above problems. First, its own random distribution weigthts are used to assure the diversity between each ELM base learner, then leave-one-out (LOO) cross validation method and M S E P R E S Smethod are used to find the optimal hidden node number of each base learner, calculate the optimal hidden layer output weights to train better and different base learners. Then the new penalty term about diversity is explicitly added to the objective function and the output matrix of each learner is updated iteratively. Finally, the final output of the whole network model is obtained by averaging the output of all base learners. This method can effectively realize the ensemble of regularized extreme learning machines (RELM) with both accuracy and diversity. Experimental results on 10 UCI datasets indicate the effectiveness of DRELM.http://fcst.ceaj.org/fileup/1673-9418/PDF/2101001.pdf|extreme learning machine (elm)|ensemble learning|diversity|regularized extreme learning machines (relm) |
| spellingShingle | CHEN Yang, WANG Shitong Ensemble Method of Diverse Regularized Extreme Learning Machines Jisuanji kexue yu tansuo |extreme learning machine (elm)|ensemble learning|diversity|regularized extreme learning machines (relm) |
| title | Ensemble Method of Diverse Regularized Extreme Learning Machines |
| title_full | Ensemble Method of Diverse Regularized Extreme Learning Machines |
| title_fullStr | Ensemble Method of Diverse Regularized Extreme Learning Machines |
| title_full_unstemmed | Ensemble Method of Diverse Regularized Extreme Learning Machines |
| title_short | Ensemble Method of Diverse Regularized Extreme Learning Machines |
| title_sort | ensemble method of diverse regularized extreme learning machines |
| topic | |extreme learning machine (elm)|ensemble learning|diversity|regularized extreme learning machines (relm) |
| url | http://fcst.ceaj.org/fileup/1673-9418/PDF/2101001.pdf |
| work_keys_str_mv | AT chenyangwangshitong ensemblemethodofdiverseregularizedextremelearningmachines |