Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks
This paper proposes a distributed estimation technique utilizing the diffusion least mean-square (LMS) algorithm, specifically designed for sparse systems in which many coefficients of the system are zeros. To efficiently utilize the sparse representation of the system and achieve a promising perfor...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/22/4638 |
| _version_ | 1827639346301239296 |
|---|---|
| author | Han-Sol Lee Changgyun Jin Chanwoo Shin Seong-Eun Kim |
| author_facet | Han-Sol Lee Changgyun Jin Chanwoo Shin Seong-Eun Kim |
| author_sort | Han-Sol Lee |
| collection | DOAJ |
| description | This paper proposes a distributed estimation technique utilizing the diffusion least mean-square (LMS) algorithm, specifically designed for sparse systems in which many coefficients of the system are zeros. To efficiently utilize the sparse representation of the system and achieve a promising performance, we have incorporated <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mn>0</mn></msub></semantics></math></inline-formula>-norm regularization into the diffusion LMS algorithm. This integration is accomplished by employing hard thresholding through a variable splitting method into the update equation. The efficacy of our approach is validated by comprehensive theoretical analysis, rigorously examining the mean stability as well as the transient and steady-state behaviors of the proposed algorithm. The proposed algorithm preserves the behavior of large coefficients and strongly enforces smaller coefficients toward zero through the relaxation of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mn>0</mn></msub></semantics></math></inline-formula>-norm regularization. Experimental results show that the proposed algorithm achieves superior convergence performance compared with conventional sparse algorithms. |
| first_indexed | 2024-03-09T16:38:13Z |
| format | Article |
| id | doaj.art-6f498bd439184ec481dfbf3d78a32947 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T16:38:13Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-6f498bd439184ec481dfbf3d78a329472023-11-24T14:54:18ZengMDPI AGMathematics2227-73902023-11-011122463810.3390/math11224638Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over NetworksHan-Sol Lee0Changgyun Jin1Chanwoo Shin2Seong-Eun Kim3System LSI Business, Samsung Electronics, Hwaseong 18448, Republic of KoreaDepartment of Applied Artificial Intelligence, Seoul National University of Science and Technology, Seoul 01811, Republic of KoreaDepartment of Applied Artificial Intelligence, Seoul National University of Science and Technology, Seoul 01811, Republic of KoreaDepartment of Applied Artificial Intelligence, Seoul National University of Science and Technology, Seoul 01811, Republic of KoreaThis paper proposes a distributed estimation technique utilizing the diffusion least mean-square (LMS) algorithm, specifically designed for sparse systems in which many coefficients of the system are zeros. To efficiently utilize the sparse representation of the system and achieve a promising performance, we have incorporated <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mn>0</mn></msub></semantics></math></inline-formula>-norm regularization into the diffusion LMS algorithm. This integration is accomplished by employing hard thresholding through a variable splitting method into the update equation. The efficacy of our approach is validated by comprehensive theoretical analysis, rigorously examining the mean stability as well as the transient and steady-state behaviors of the proposed algorithm. The proposed algorithm preserves the behavior of large coefficients and strongly enforces smaller coefficients toward zero through the relaxation of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mn>0</mn></msub></semantics></math></inline-formula>-norm regularization. Experimental results show that the proposed algorithm achieves superior convergence performance compared with conventional sparse algorithms.https://www.mdpi.com/2227-7390/11/22/4638diffusion least mean squaredistributed estimationsparse parametersystem identificationhard thresholding |
| spellingShingle | Han-Sol Lee Changgyun Jin Chanwoo Shin Seong-Eun Kim Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks Mathematics diffusion least mean square distributed estimation sparse parameter system identification hard thresholding |
| title | Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks |
| title_full | Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks |
| title_fullStr | Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks |
| title_full_unstemmed | Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks |
| title_short | Sparse Diffusion Least Mean-Square Algorithm with Hard Thresholding over Networks |
| title_sort | sparse diffusion least mean square algorithm with hard thresholding over networks |
| topic | diffusion least mean square distributed estimation sparse parameter system identification hard thresholding |
| url | https://www.mdpi.com/2227-7390/11/22/4638 |
| work_keys_str_mv | AT hansollee sparsediffusionleastmeansquarealgorithmwithhardthresholdingovernetworks AT changgyunjin sparsediffusionleastmeansquarealgorithmwithhardthresholdingovernetworks AT chanwooshin sparsediffusionleastmeansquarealgorithmwithhardthresholdingovernetworks AT seongeunkim sparsediffusionleastmeansquarealgorithmwithhardthresholdingovernetworks |