Sparsity Increases Uncertainty Estimation in Deep Ensemble
Deep neural networks have achieved almost human-level results in various tasks and have become popular in the broad artificial intelligence domains. Uncertainty estimation is an on-demand task caused by the black-box point estimation behavior of deep learning. The deep ensemble provides increased ac...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Computers |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-431X/10/4/54 |
| _version_ | 1797537094517653504 |
|---|---|
| author | Uyanga Dorjsembe Ju Hong Lee Bumghi Choi Jae Won Song |
| author_facet | Uyanga Dorjsembe Ju Hong Lee Bumghi Choi Jae Won Song |
| author_sort | Uyanga Dorjsembe |
| collection | DOAJ |
| description | Deep neural networks have achieved almost human-level results in various tasks and have become popular in the broad artificial intelligence domains. Uncertainty estimation is an on-demand task caused by the black-box point estimation behavior of deep learning. The deep ensemble provides increased accuracy and estimated uncertainty; however, linearly increasing the size makes the deep ensemble unfeasible for memory-intensive tasks. To address this problem, we used model pruning and quantization with a deep ensemble and analyzed the effect in the context of uncertainty metrics. We empirically showed that the ensemble members’ disagreement increases with pruning, making models sparser by zeroing irrelevant parameters. Increased disagreement im-plies increased uncertainty, which helps in making more robust predictions. Accordingly, an energy-efficient compressed deep ensemble is appropriate for memory-intensive and uncertainty-aware tasks. |
| first_indexed | 2024-03-10T12:10:10Z |
| format | Article |
| id | doaj.art-00171b085dbd426d8577b2047c96ebe5 |
| institution | Directory Open Access Journal |
| issn | 2073-431X |
| language | English |
| last_indexed | 2024-03-10T12:10:10Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Computers |
| spelling | doaj.art-00171b085dbd426d8577b2047c96ebe52023-11-21T16:14:34ZengMDPI AGComputers2073-431X2021-04-011045410.3390/computers10040054Sparsity Increases Uncertainty Estimation in Deep EnsembleUyanga Dorjsembe0Ju Hong Lee1Bumghi Choi2Jae Won Song3Department of Computer Science, Inha University, 100 Inha-ro, Michuhol-gu, Incheon 22212, KoreaDepartment of Computer Science, Inha University, 100 Inha-ro, Michuhol-gu, Incheon 22212, KoreaQHedge Inc, Inha Dream Center, 100 Inha-ro, Michuhol-gu, Incheon 22212, KoreaValue finders Inc, Incheon IT Tower, 229 Gyeongin-ro, Michuhol-gu, Incheon 22106, KoreaDeep neural networks have achieved almost human-level results in various tasks and have become popular in the broad artificial intelligence domains. Uncertainty estimation is an on-demand task caused by the black-box point estimation behavior of deep learning. The deep ensemble provides increased accuracy and estimated uncertainty; however, linearly increasing the size makes the deep ensemble unfeasible for memory-intensive tasks. To address this problem, we used model pruning and quantization with a deep ensemble and analyzed the effect in the context of uncertainty metrics. We empirically showed that the ensemble members’ disagreement increases with pruning, making models sparser by zeroing irrelevant parameters. Increased disagreement im-plies increased uncertainty, which helps in making more robust predictions. Accordingly, an energy-efficient compressed deep ensemble is appropriate for memory-intensive and uncertainty-aware tasks.https://www.mdpi.com/2073-431X/10/4/54deep learninguncertainty estimationdeep ensemblemodel compression |
| spellingShingle | Uyanga Dorjsembe Ju Hong Lee Bumghi Choi Jae Won Song Sparsity Increases Uncertainty Estimation in Deep Ensemble Computers deep learning uncertainty estimation deep ensemble model compression |
| title | Sparsity Increases Uncertainty Estimation in Deep Ensemble |
| title_full | Sparsity Increases Uncertainty Estimation in Deep Ensemble |
| title_fullStr | Sparsity Increases Uncertainty Estimation in Deep Ensemble |
| title_full_unstemmed | Sparsity Increases Uncertainty Estimation in Deep Ensemble |
| title_short | Sparsity Increases Uncertainty Estimation in Deep Ensemble |
| title_sort | sparsity increases uncertainty estimation in deep ensemble |
| topic | deep learning uncertainty estimation deep ensemble model compression |
| url | https://www.mdpi.com/2073-431X/10/4/54 |
| work_keys_str_mv | AT uyangadorjsembe sparsityincreasesuncertaintyestimationindeepensemble AT juhonglee sparsityincreasesuncertaintyestimationindeepensemble AT bumghichoi sparsityincreasesuncertaintyestimationindeepensemble AT jaewonsong sparsityincreasesuncertaintyestimationindeepensemble |