Feature Learning Viewpoint of Adaboost and a New Algorithm
The AdaBoost algorithm has the superiority of resisting overfitting. Understanding the mysteries of this phenomenon is a very fascinating fundamental theoretical problem. Many studies are devoted to explaining it from statistical view and margin theory. In this paper, this phenomenon is illustrated...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8868178/ |
| _version_ | 1798001954856632320 |
|---|---|
| author | Fei Wang Zhongheng Li Fang He Rong Wang Weizhong Yu Feiping Nie |
| author_facet | Fei Wang Zhongheng Li Fang He Rong Wang Weizhong Yu Feiping Nie |
| author_sort | Fei Wang |
| collection | DOAJ |
| description | The AdaBoost algorithm has the superiority of resisting overfitting. Understanding the mysteries of this phenomenon is a very fascinating fundamental theoretical problem. Many studies are devoted to explaining it from statistical view and margin theory. In this paper, this phenomenon is illustrated by the proposed AdaBoost+SVM algorithm from feature learning viewpoint, which clearly explains the resistance to overfitting of AdaBoost. Firstly, we adopt the AdaBoost algorithm to learn the base classifiers. Then, instead of directly combining the base classifiers, we regard them as features and input them to SVM classifier. With this, the new coefficient and bias can be obtained, which can be used to construct the final classifier. We explain the rationality of this and illustrate the theorem that when the dimension of these features increases, the performance of SVM would not be worse, which can explain the resistance to overfitting of AdaBoost. |
| first_indexed | 2024-04-11T11:44:29Z |
| format | Article |
| id | doaj.art-706f9831d58f48a3af275dd6733d345b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T11:44:29Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-706f9831d58f48a3af275dd6733d345b2022-12-22T04:25:40ZengIEEEIEEE Access2169-35362019-01-01714989014989910.1109/ACCESS.2019.29473598868178Feature Learning Viewpoint of Adaboost and a New AlgorithmFei Wang0Zhongheng Li1https://orcid.org/0000-0001-7091-9600Fang He2Rong Wang3https://orcid.org/0000-0001-9240-6726Weizhong Yu4Feiping Nie5National Engineering Laboratory for Visual Information Processing and Applications, Xi’an Jiaotong University, Xi’an, ChinaNational Engineering Laboratory for Visual Information Processing and Applications, Xi’an Jiaotong University, Xi’an, ChinaXi’an Research Institute of Hi-Tech, Xi’an, ChinaCenter for Optical Imagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, ChinaNational Engineering Laboratory for Visual Information Processing and Applications, Xi’an Jiaotong University, Xi’an, ChinaCenter for Optical Imagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, ChinaThe AdaBoost algorithm has the superiority of resisting overfitting. Understanding the mysteries of this phenomenon is a very fascinating fundamental theoretical problem. Many studies are devoted to explaining it from statistical view and margin theory. In this paper, this phenomenon is illustrated by the proposed AdaBoost+SVM algorithm from feature learning viewpoint, which clearly explains the resistance to overfitting of AdaBoost. Firstly, we adopt the AdaBoost algorithm to learn the base classifiers. Then, instead of directly combining the base classifiers, we regard them as features and input them to SVM classifier. With this, the new coefficient and bias can be obtained, which can be used to construct the final classifier. We explain the rationality of this and illustrate the theorem that when the dimension of these features increases, the performance of SVM would not be worse, which can explain the resistance to overfitting of AdaBoost.https://ieeexplore.ieee.org/document/8868178/AdaBoostfeature learningoverfittingSVM |
| spellingShingle | Fei Wang Zhongheng Li Fang He Rong Wang Weizhong Yu Feiping Nie Feature Learning Viewpoint of Adaboost and a New Algorithm IEEE Access AdaBoost feature learning overfitting SVM |
| title | Feature Learning Viewpoint of Adaboost and a New Algorithm |
| title_full | Feature Learning Viewpoint of Adaboost and a New Algorithm |
| title_fullStr | Feature Learning Viewpoint of Adaboost and a New Algorithm |
| title_full_unstemmed | Feature Learning Viewpoint of Adaboost and a New Algorithm |
| title_short | Feature Learning Viewpoint of Adaboost and a New Algorithm |
| title_sort | feature learning viewpoint of adaboost and a new algorithm |
| topic | AdaBoost feature learning overfitting SVM |
| url | https://ieeexplore.ieee.org/document/8868178/ |
| work_keys_str_mv | AT feiwang featurelearningviewpointofadaboostandanewalgorithm AT zhonghengli featurelearningviewpointofadaboostandanewalgorithm AT fanghe featurelearningviewpointofadaboostandanewalgorithm AT rongwang featurelearningviewpointofadaboostandanewalgorithm AT weizhongyu featurelearningviewpointofadaboostandanewalgorithm AT feipingnie featurelearningviewpointofadaboostandanewalgorithm |