A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost
This paper proposes a new multi-kernel learning ensemble algorithm, called Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR, which can be regarded as an extension of multi-kernel learning (MKL) and weighted support vector...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9714461/ |
| _version_ | 1818504725984706560 |
|---|---|
| author | Xiaojin Xie Kangyang Luo Guoqiang Wang |
| author_facet | Xiaojin Xie Kangyang Luo Guoqiang Wang |
| author_sort | Xiaojin Xie |
| collection | DOAJ |
| description | This paper proposes a new multi-kernel learning ensemble algorithm, called Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR, which can be regarded as an extension of multi-kernel learning (MKL) and weighted support vector regression (WSVR). The first novelty is to add the <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula> norm of the weights of the combined kernel function to the objective function of WSVR, which is used to adaptively select the optimal base models and their parameters. In addition, an accelerated method based on fast iterative shrinkage thresholding algorithm (FISTA) is developed to solve the weights of the combined kernel function. The second novelty is to propose an integrated learning framework based on AdaBoost, named Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR. In this framework, we integrate FISTA into AdaBoost. At each iteration, we optimize the weights of the combined kernel function and update the weights of the training samples at the same time. Then an ensemble regression function of a set of regression functions is output. Finally, two groups of the experiments are designed to verify the performance of our algorithm. On the first group of the experiments including eight datasets from UCI machine learning repository, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 11.14% and 9.08% on average, respectively. Furthermore, on the second group of the experiments including the COVID-19 epidemic datasets from eight countries, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 31.19% and 29.98% on average, respectively. |
| first_indexed | 2024-12-10T21:41:05Z |
| format | Article |
| id | doaj.art-8bf1674a1ebc416ea9c1ec35363936c3 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T21:41:05Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-8bf1674a1ebc416ea9c1ec35363936c32022-12-22T01:32:31ZengIEEEIEEE Access2169-35362022-01-0110203752038410.1109/ACCESS.2022.31516729714461A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoostXiaojin Xie0Kangyang Luo1https://orcid.org/0000-0003-0123-061XGuoqiang Wang2https://orcid.org/0000-0003-2979-3510School of Mathematics, Physics and Statistics, Shanghai University of Engineering Science, Shanghai, ChinaSchool of Data Science and Engineering, East China Normal University, Shanghai, ChinaSchool of Mathematics, Physics and Statistics, Shanghai University of Engineering Science, Shanghai, ChinaThis paper proposes a new multi-kernel learning ensemble algorithm, called Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR, which can be regarded as an extension of multi-kernel learning (MKL) and weighted support vector regression (WSVR). The first novelty is to add the <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula> norm of the weights of the combined kernel function to the objective function of WSVR, which is used to adaptively select the optimal base models and their parameters. In addition, an accelerated method based on fast iterative shrinkage thresholding algorithm (FISTA) is developed to solve the weights of the combined kernel function. The second novelty is to propose an integrated learning framework based on AdaBoost, named Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR. In this framework, we integrate FISTA into AdaBoost. At each iteration, we optimize the weights of the combined kernel function and update the weights of the training samples at the same time. Then an ensemble regression function of a set of regression functions is output. Finally, two groups of the experiments are designed to verify the performance of our algorithm. On the first group of the experiments including eight datasets from UCI machine learning repository, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 11.14% and 9.08% on average, respectively. Furthermore, on the second group of the experiments including the COVID-19 epidemic datasets from eight countries, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 31.19% and 29.98% on average, respectively.https://ieeexplore.ieee.org/document/9714461/Support vector regressionmulti-kernel learningAdaBoostensemble algorithmregression prediction |
| spellingShingle | Xiaojin Xie Kangyang Luo Guoqiang Wang A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost IEEE Access Support vector regression multi-kernel learning AdaBoost ensemble algorithm regression prediction |
| title | A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost |
| title_full | A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost |
| title_fullStr | A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost |
| title_full_unstemmed | A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost |
| title_short | A New <italic>L</italic>₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost |
| title_sort | new italic l italic x2081 multi kernel learning support vector regression ensemble algorithm with adaboost |
| topic | Support vector regression multi-kernel learning AdaBoost ensemble algorithm regression prediction |
| url | https://ieeexplore.ieee.org/document/9714461/ |
| work_keys_str_mv | AT xiaojinxie anewitaliclitalicx2081multikernellearningsupportvectorregressionensemblealgorithmwithadaboost AT kangyangluo anewitaliclitalicx2081multikernellearningsupportvectorregressionensemblealgorithmwithadaboost AT guoqiangwang anewitaliclitalicx2081multikernellearningsupportvectorregressionensemblealgorithmwithadaboost AT xiaojinxie newitaliclitalicx2081multikernellearningsupportvectorregressionensemblealgorithmwithadaboost AT kangyangluo newitaliclitalicx2081multikernellearningsupportvectorregressionensemblealgorithmwithadaboost AT guoqiangwang newitaliclitalicx2081multikernellearningsupportvectorregressionensemblealgorithmwithadaboost |