An Adaptive Learning Algorithm for Regularized Extreme Learning Machine
Extreme learning machine (ELM) has become popular in recent years, due to its robust approximation capacity and fast learning speed. It is common to add a <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula> penalty term in basic ELM...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9335603/ |
| _version_ | 1811274490466795520 |
|---|---|
| author | Yuao Zhang Qingbiao Wu Jueliang Hu |
| author_facet | Yuao Zhang Qingbiao Wu Jueliang Hu |
| author_sort | Yuao Zhang |
| collection | DOAJ |
| description | Extreme learning machine (ELM) has become popular in recent years, due to its robust approximation capacity and fast learning speed. It is common to add a <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula> penalty term in basic ELM to avoid over-fitting. However, in <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula>-regularized extreme learning machine (<inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula>-RELM), choosing a suitable regularization factor is random and time consuming. In order to select a satisfactory regularization factor automatically, we proposed an adaptive regularized extreme learning machine (A-RELM) by replacing the regularization factor with a function. The function is defined in terms of the output weights named regularization function. And an iterative algorithm is proposed for obtaining the output weights, therefore, allowing for deriving their values simultaneously. Besides, the constructed regularization function ensures the convexity of the model, which contributes to a globally optimal solution. The convergence analysis of the iterative algorithm guarantees the effectiveness of the model training. Experimental results on some UCI benchmarks and the Yale face database B indicate the superiority of our proposed algorithm. |
| first_indexed | 2024-04-12T23:20:01Z |
| format | Article |
| id | doaj.art-fafb18297010498186406bc0b517cc55 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T23:20:01Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-fafb18297010498186406bc0b517cc552022-12-22T03:12:33ZengIEEEIEEE Access2169-35362021-01-019207362074510.1109/ACCESS.2021.30544839335603An Adaptive Learning Algorithm for Regularized Extreme Learning MachineYuao Zhang0https://orcid.org/0000-0002-8664-9788Qingbiao Wu1https://orcid.org/0000-0003-2706-6264Jueliang Hu2https://orcid.org/0000-0003-0763-536XDepartment of Mathematics, Zhejiang University, Hangzhou, ChinaDepartment of Mathematics, Zhejiang University, Hangzhou, ChinaDepartment of Mathematics, Zhejiang Sci-Tech University, Hangzhou, ChinaExtreme learning machine (ELM) has become popular in recent years, due to its robust approximation capacity and fast learning speed. It is common to add a <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula> penalty term in basic ELM to avoid over-fitting. However, in <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula>-regularized extreme learning machine (<inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula>-RELM), choosing a suitable regularization factor is random and time consuming. In order to select a satisfactory regularization factor automatically, we proposed an adaptive regularized extreme learning machine (A-RELM) by replacing the regularization factor with a function. The function is defined in terms of the output weights named regularization function. And an iterative algorithm is proposed for obtaining the output weights, therefore, allowing for deriving their values simultaneously. Besides, the constructed regularization function ensures the convexity of the model, which contributes to a globally optimal solution. The convergence analysis of the iterative algorithm guarantees the effectiveness of the model training. Experimental results on some UCI benchmarks and the Yale face database B indicate the superiority of our proposed algorithm.https://ieeexplore.ieee.org/document/9335603/Adaptiveconvergenceconvexityextreme learning machine (ELM)regularization |
| spellingShingle | Yuao Zhang Qingbiao Wu Jueliang Hu An Adaptive Learning Algorithm for Regularized Extreme Learning Machine IEEE Access Adaptive convergence convexity extreme learning machine (ELM) regularization |
| title | An Adaptive Learning Algorithm for Regularized Extreme Learning Machine |
| title_full | An Adaptive Learning Algorithm for Regularized Extreme Learning Machine |
| title_fullStr | An Adaptive Learning Algorithm for Regularized Extreme Learning Machine |
| title_full_unstemmed | An Adaptive Learning Algorithm for Regularized Extreme Learning Machine |
| title_short | An Adaptive Learning Algorithm for Regularized Extreme Learning Machine |
| title_sort | adaptive learning algorithm for regularized extreme learning machine |
| topic | Adaptive convergence convexity extreme learning machine (ELM) regularization |
| url | https://ieeexplore.ieee.org/document/9335603/ |
| work_keys_str_mv | AT yuaozhang anadaptivelearningalgorithmforregularizedextremelearningmachine AT qingbiaowu anadaptivelearningalgorithmforregularizedextremelearningmachine AT juelianghu anadaptivelearningalgorithmforregularizedextremelearningmachine AT yuaozhang adaptivelearningalgorithmforregularizedextremelearningmachine AT qingbiaowu adaptivelearningalgorithmforregularizedextremelearningmachine AT juelianghu adaptivelearningalgorithmforregularizedextremelearningmachine |