Shapley Values as a Strategy for Ensemble Weights Estimation
This study introduces a novel performance-based weighting scheme for ensemble learning using the Shapley value. The weighting uses the reciprocal of binary cross-entropy as a base learner’s performance metric and estimates its Shapley value to measure the overall contribution of a learner to an equa...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/12/7010 |
| _version_ | 1827738768062283776 |
|---|---|
| author | Vaidotas Drungilas Evaldas Vaičiukynas Linas Ablonskis Lina Čeponienė |
| author_facet | Vaidotas Drungilas Evaldas Vaičiukynas Linas Ablonskis Lina Čeponienė |
| author_sort | Vaidotas Drungilas |
| collection | DOAJ |
| description | This study introduces a novel performance-based weighting scheme for ensemble learning using the Shapley value. The weighting uses the reciprocal of binary cross-entropy as a base learner’s performance metric and estimates its Shapley value to measure the overall contribution of a learner to an equally weighted ensemble of various sizes. Two variants of this strategy were empirically compared with a single monolith model and other static weighting strategies using two large banking-related datasets. A variant that discards learners with a negative Shapley value was ranked as first or at least second when constructing homogeneous ensembles, whereas for heterogeneous ensembles this strategy resulted in a better or at least similar detection performance to other weighting strategies tested. The main limitation being the computational complexity of Shapley calculations, the explored weighting strategy could be considered as a generalization of performance-based weighting. |
| first_indexed | 2024-03-11T02:48:58Z |
| format | Article |
| id | doaj.art-a48081a78458412a8573ad9899531bdf |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T02:48:58Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-a48081a78458412a8573ad9899531bdf2023-11-18T09:07:42ZengMDPI AGApplied Sciences2076-34172023-06-011312701010.3390/app13127010Shapley Values as a Strategy for Ensemble Weights EstimationVaidotas Drungilas0Evaldas Vaičiukynas1Linas Ablonskis2Lina Čeponienė3Department of Information Systems, Faculty of Informatics, Kaunas University of Technology, 44249 Kaunas, LithuaniaDepartment of Information Systems, Faculty of Informatics, Kaunas University of Technology, 44249 Kaunas, LithuaniaDepartment of Information Systems, Faculty of Informatics, Kaunas University of Technology, 44249 Kaunas, LithuaniaDepartment of Information Systems, Faculty of Informatics, Kaunas University of Technology, 44249 Kaunas, LithuaniaThis study introduces a novel performance-based weighting scheme for ensemble learning using the Shapley value. The weighting uses the reciprocal of binary cross-entropy as a base learner’s performance metric and estimates its Shapley value to measure the overall contribution of a learner to an equally weighted ensemble of various sizes. Two variants of this strategy were empirically compared with a single monolith model and other static weighting strategies using two large banking-related datasets. A variant that discards learners with a negative Shapley value was ranked as first or at least second when constructing homogeneous ensembles, whereas for heterogeneous ensembles this strategy resulted in a better or at least similar detection performance to other weighting strategies tested. The main limitation being the computational complexity of Shapley calculations, the explored weighting strategy could be considered as a generalization of performance-based weighting.https://www.mdpi.com/2076-3417/13/12/7010machine learningensemble methodsShapley valueperformance weightingprivacy-preserving distributed learning |
| spellingShingle | Vaidotas Drungilas Evaldas Vaičiukynas Linas Ablonskis Lina Čeponienė Shapley Values as a Strategy for Ensemble Weights Estimation Applied Sciences machine learning ensemble methods Shapley value performance weighting privacy-preserving distributed learning |
| title | Shapley Values as a Strategy for Ensemble Weights Estimation |
| title_full | Shapley Values as a Strategy for Ensemble Weights Estimation |
| title_fullStr | Shapley Values as a Strategy for Ensemble Weights Estimation |
| title_full_unstemmed | Shapley Values as a Strategy for Ensemble Weights Estimation |
| title_short | Shapley Values as a Strategy for Ensemble Weights Estimation |
| title_sort | shapley values as a strategy for ensemble weights estimation |
| topic | machine learning ensemble methods Shapley value performance weighting privacy-preserving distributed learning |
| url | https://www.mdpi.com/2076-3417/13/12/7010 |
| work_keys_str_mv | AT vaidotasdrungilas shapleyvaluesasastrategyforensembleweightsestimation AT evaldasvaiciukynas shapleyvaluesasastrategyforensembleweightsestimation AT linasablonskis shapleyvaluesasastrategyforensembleweightsestimation AT linaceponiene shapleyvaluesasastrategyforensembleweightsestimation |